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INTRODUCTION 



A. BACKGROUND 

The Navy is currently testing a new source selection tool 
known as Red Yellow Green (RYG) . The test is being conducted 
at five Navy field activities (Naval Air Engineering Center 
Lakehurst, Naval Avionics Center Indianapolis, Naval Ships 
Parts Control Center Mechanicsburg (Code 021, Level 1/SS) , 
Naval Supply Center Charleston/Naval Shipyard Charleston, and 
Naval Supply Center Pensacola/Naval Aviation Depot Pensacola) . 
The purpose of the RYG program is to assist Government 
contracting personnel in obtaining the best purchase value by 
adding the costs of poor contractor performance into the 
source selection equation [Ref. 1]. The RYG program was 
designed especially for use by field contracting activities 
[Ref. 2]. 

In addition to RYG, several other contractor quality 
programs are currently being employed or developed within the 
Department of Defense (DOD) that consider contractor quality 
and past performance in the source selection process. Three 
notable quality programs are the Blue Ribbon Contractor 
program, the Contractor Performance Assessment Reporting 
System (CPARS) , and Total Quality Management (TQM) . This 
thesis will describe the four programs listed above. It will 
then evaluate the RYG program by comparing and contrasting it 
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to the other three programs, discussing its strengths and 
weaknesses, and describing the status of the program test. 

B. OBJECTIVES 

DOD and its agencies have continuously sought ways to 
improve the quality of Government contractor performance while 
complying with the statutes, regulations, and policies that 
govern the Federal acquisition process. Recent problems with 
the quality of material delivered to DOD, many of them 
highlighted in the press, have increased DOD's emphasis on 
contractor quality management. At the same time, steadily 
increasing Congressional oversight in the form of more and 
increasingly detailed statutes and guidance have made 
development of a viable program more difficult. Some of the 
programs currently employed or under review by DOD commands 
include the Navy's RYG program, Blue Ribbon Contractor 
programs within the Navy and Air Force, the Air Force's CPARS , 
and the DOD-wide TQM initiative. 

One of these programs, the Navy's RYG, is the subject of 
this thesis. This program is intended to eventually be 
extended to all DOD agencies. It uses the data contained in 
the Navy's newly established unified, centralized Contractor 
Evaluation System (CES) . These data are compiled from many 
sources including Quality Deficiency Reports (QDRs) and the 
Naval Material Quality Assessment Office (NMQAO) . [Ref. 3] 
Prior to CES, these data were found in many different forms 
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and each came from a different source. Now, they are 
available in a single location and can be accessed by field 
procurement activities from a computer terminal using an on- 
line call-up menu. CES is used to classify contractors (by 
Federal Supply Class (FSC) ) as high risk quality performers 
(Red) , moderate risk (Yellow) , or low risk (Green) . 
Contractors who do not meet established criteria for RYG 
classification are listed in the "Insufficient Data" 
category. [Ref. l:p. 1] The RYG classification is used by 
contracting personnel to either apply Technical Evaluation 
Adjustments (TEAs) to contractor proposals [Ref. l:p. 1] or, 
if the Fixed Price-Greatest Value method is employed, to rate 
the offerors in terms of expected quality of performance [Ref. 
l:encl (3) , p. 3]. The TEAs represent the anticipated cost to 
the Government to correct or take appropriate action due to 
poor contractor performance [Ref. l:encl (1), pp. 1-2]. 
Proposals including TEAs are used to help determine a source 
selection that will result in the best overall price to the 
Government [Ref. l:encl (1), p. 6]. 

The RYG program is undergoing a one-year test coordinated 
by the Office of Assistant Secretary of the Navy (Shipbuilding 
and Logistics) Reliability, Maintainability, and Quality 
Assurance (ASN (S&L) RM&QA) which began 1 August 1989 using a 
limited number of commodities. 



3 



C. THE RESEARCH QUESTIONS 

The primary research question is: How is the Navy's Red 

Yellow Green program structured and how is it intended to 
improve the quality of material procured by the Navy? The 
subsidiary research questions are: 

1. How are Red, Yellow, and Green contractors defined? 

2. What are the other current contractor quality 

initiatives within DOD? 

3 . How does RYG compare and contrast to these other 
programs? 

4. What are the strengths and weaknesses associated with 
RYG? 

5. How is the RYG program being tested in the market? 

D. SCOPE AND LIMITATIONS 

The purpose of this thesis is to evaluate the potential 
effectiveness of the Navy's RYG program. It will analyze the 
program in detail and describe other DOD contractor quality 
initiatives in brief. It will compare and contrast the RYG 
program to the other DOD initiatives and will attempt to 
identify RYG strengths and weaknesses. It will evaluate the 
success of RYG tests as far as they have progressed. Finally, 
it will consider the viability of the RYG program in the 
future, both within the Navy and DOD. 

The research effort was limited by the short period of 
time over which the test of RYG could be evaluated. Although 
some test activities were able to begin implementation of the 
RYG program on 1 August 1989, others were delayed. This 
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thesis evaluated the program very early in the test period, 
and, therefore, relied on limited data currently available. 

E. METHODOLOGY 

Two methods were used to collect data and answer the 
research questions. Data were collected from secondary 
sources to obtain information about RYG and other DOD contrac- 
tor quality initiatives. Data were collected from primary 
sources, specifically through telephone and personal inter- 
views, to add to the above areas and to assess the current 
status of the test and the potential for future expansion of 
the program. The interviews involved personnel at 

ASN (S&L) RM&QA, NMQAO , the Naval Supply Systems Command, and 
the five activities involved in the test. 



F. ORGANIZATION OF THE STUDY 

The remainder of the thesis is organized into the follow- 
ing chapters : 

1. Chapter II, "Red Yellow Green Program," will describe 
the RYG program concept and how it is being coordinated. 

2. Chapter III, "Other Department of Defense Quality 
Initiatives," will briefly describe other current DOD 
contractor quality programs in place or under review. 

3. Chapter IV, "A comparison of Red Yellow Green and Other 
Contractor Quality Initiatives," will compare and 
contrast the RYG program to the quality programs 
described in Chapter III. 

4. Chapter V, "Strengths and Weaknesses of Red Yellow 
Green," will describe the advantages and potential 
problems and limitations associated with RYG. It will 
also discuss the status of the RYG test, describing how 
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the program test is progressing and attempting to 
project how the program will fare in the future. 

5. Chapter VI, "Conclusions and Recommendations," will 
discuss whether RYG will be effective in improving the 
quality of material procured by the Navy and will 
comment on possible changes for weak areas of the 
program. 
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II . RED YELLOW GREEN PROGRAM 



A . BACKGROUND 

The Navy's Contractor Evaluation System (CES) and Red 
Yellow Green (RYG) program were established to both assist 
Navy activities in procuring quality goods and services and to 
comply with Department of Defense (DOD) and Secretary of the 
Navy ( SECNAV) policy. Specifically, DOD Directive 4155.1 
states : 

DOD Components shall assure that contracts are not awarded 
to contractors with a previous history of providing products 
or services of an unsatisfactory quality. Contractor 
quality history data shall be maintained and used for this 
purpose. [Ref. 3:p. 1-1] 

Further, SECNAV Instruction 4855.7 states: 

It is Secretary of the Navy policy that: 

a. Contractor quality history data be collected and 
maintained in a single Navy CES. 

b. Contractor quality history data be used to assure that 
contracts are not awarded to contractors with a previous 
history of providing products of an unsatisfactory quality, 
and to determine actions necessary before and after contract 
awards to assure product quality. [Ref. 4] 

The CES and RYG program satisfy the requirements of these 

policies . 



B. CONTRACTOR EVALUATION SYSTEM 

The CES is currently composed of data from the Product 
Deficiency Reporting and Evaluation Program (PDREP) data base. 
The Navy has plans to expand the CES to include other general 
information about contractors such as financial data in 
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addition to information concerning contractor quality. This 
expanded CES will be called the Buyer Information System. It 
will be available to Government contracting personnel to help 
them better evaluate offerors. At present, however, the 
CES/PDREP is a quality reporting system managed by the Naval 
Sea Systems Command (NAVSEA) Detachment, Naval Material 
Quality Assessment Office (NMQAO) , under the direction of the 
Office of the Assistant Secretary of the Navy (Shipbuilding 
and Logistics) Reliability, Maintainability, and Quality 
Assurance (ASN (S&L) RM&QA) . [Ref. 3:p. v] 

PDREP was established to satisfy the requirements of 
SECNAV Instruction 4855. 1A for "a product deficiency reporting 
and data feedback system, maintenance of contractor/supplier 
quality history and effective use of the data to influence the 
pre-contract award process and formulate the basis for 
necessary post award quality assurance actions." [Ref. 5] 
The purpose of PDREP is to 

...implement a product deficiency reporting and evaluation 
program that will feed data back to those activities respon- 
sible for design, development, purchasing, production, 
supply, maintenance, contract administration, and other 
acquisition functions and initiate action to appraise, 
correct, and prevent product deficiencies, decrease material 
ownership costs, evaluate contractor/supplier performance 
and use performance data for making procurement decisions. 
[Ref. 5 : p . 2] 

PDREP' s purpose, then, is compatible with the goals of the RYG 
program. Therefore, it was a logical data base to be chosen 
for the Navy's new CES which was intended to be used to 
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categorize contractors as Red, Yellow, or Green within a given 
Federal Supply Class (FSC) . 

The CES is composed of contractor quality information 
collected from the sources which are described below. 

1. Quality Deficiency Reports (QDRs) . QDRs are prepared by 
Navy field activities to document product quality 
deficiencies, design deficiencies, or inadequate 
procurement documents resulting in defective new and 
newly reworked material being delivered to the Navy 
[Ref. 6]. All QDRs are submitted to Naval Air Systems 
Command (NAVAIR) , the Navy focal point for QDRs. Once 
each week QDRs determined to be contractor liable and 
with defects verified are transmitted by NAVAIR to NMQAO 
for inclusion in PDREP . [Ref. 7] 

2. Material Inspection Record (MIR). MIRs are prepared 

either by Navy representatives performing technical 
inspections at a contractor's plant or by Navy field 
activities performing technical inspections upon receipt 
of material. MIRs are submitted to the Navy Systems 
Command having cognizance over the field activity. 
[Ref. 3:p. 1-5] The Systems Commands (NAVAIR, NAVSUP, 

Naval Facilities Engineering Command, and Space and 
Naval Warfare Command) then transmit the MIRs to NMQAO. 
The MIRs generated by NAVSEA activities are submitted 
directly to NMQAO. [Ref. 7] 

3. Reports of Discrepancy (RODs) . RODs are prepared by 

Navy field activities to document receipt of incorrect 
material, shortages and overages, and discrepancies in 
preservation, packing, and marking [Ref. 6:p. 1-2]. 

RODs are submitted to Naval Supply Systems Command 
(NAVSUP) . Procedures are currently being developed to 
transmit RODs from NAVSUP to NMQAO for inclusion in 
PDREP. However, at present they are not being 

transmitted and, therefore, are not yet part of PDREP. 
[Ref. 7] 

4. Defense Logistics Agency (DLA) Contractor Improvement 
Program (CIP) Alert List. Contractors are placed on the 
DLA alert list if DLA has placed them in the CIP, if 
they have received a negative pre-award survey, or if 
Defense Contract Administration Services (DCAS) has 
recommended they be given a pre-award survey for a 
particular reason [Ref. 8]. DLA sends a hard copy of 
the list to ASN (S&L) RM&QA. ASN then sends a copy to 
NMQAO. [Ref. 7] 
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DLA Method C, D, and E Corrective Action Listing. 
Contractors are placed on the corrective action listing 
if DLA has documented deficiencies in their quality 
programs. Specifically, method C indicates that the 
contractor has a serious quality problem or has not 
corrected a deficiency documented using method B (a 
major deficiency) . The Government sends a letter to the 
firm's top management requesting corrective action. 
Method D indicates that less severe methods of 
corrective action (i.e., A, B, and C) have failed. The 
acquisition quality assurance program is discontinued, 
and the contractor is advised that the Government will 
not accept his goods or services until deficiencies have 
been corrected. Method E is used to advise a prime 
contractor that a subcontractor has quality deficiencies 
that would justify method C or D corrective action in a 
prime contractor and to request that the prime take 
corrective action with his subcontractor. [Ref. 9] DLA 
sends a hard copy of the listing to ASN (S&L) RM&QA. 
ASN then sends a copy to NMQAO. [Ref. 7] 

6. Navy Vendor Data Analysis Report (VDAR) . The VDAR 
identifies contractors who, because of past poor 
performance, should be considered carefully before being 
awarded a contract and should be monitored after 
contract award. Evaluation of performance is based on 
data from pre-award surveys, product oriented surveys, 
quality system reviews, QDRs , open DLA method C, D, or 
E corrective action, and conviction of or investigation 
for malpractice or fraud. [Ref. 8:p. E-5] The VDAR is 
compiled by NMQAO based on past performance and input 
from Navy Systems Commands and their field activities 
[Ref. 10]. 

7. Pre-Award Surveys. Pre-award surveys are conducted by 

contract administration offices when a procuring 
contracting officer needs additional information to 
determine contractor responsibility. The survey 

evaluates the contractor's management, financial 
capability, and technical capability to determine 
whether he will be able to perform the proposed 
contract. [Ref. 11] Only those pre-award surveys 
requested by Navy activities are included in PDREP. The 
Navy activity that requested the survey submits a copy 
to the cognizant Systems Command. The Systems Commands 
then transmit copies to NMQAO. [Ref. 10] NAVSEA 
activities submit copies of surveys directly to NMQAO 
[Ref. 7]. 

8. Product-Oriented Surveys. Product-oriented surveys are 
technical product inspections conducted in a 
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contractor's plant when a buying activity desires to 
perforin a special test on an item. They are performed 
by DCAS when requested by the buying activity. If DCAS 
does not have the necessary resources, the buying 
activity may perform the survey. When a Navy activity 
requests a product-oriented survey, it submits a copy to 
the appropriate Systems Command. The Systems Commands 
then transmit the surveys to NMQAO . [Ref. 10] NAVSEA 
activities submit copies of surveys directly to NMQAO 
[Ref. 7]. 

9. Quality System Reviews. Quality system reviews are 
performed by DCAS. They involve an evaluation of the 
contractor's quality procedures and verification that 
the contractor's quality practices conform to those 
procedures. [Ref. 9:p. 23] The reviews also evaluate 

the Government's in-plant quality assurance program. 
Navy activities receive copies of quality system reviews 
if they participate in the review with DCAS or if they 
request a copy. [Ref. 10] Copies received by Navy 
activities are submitted to the appropriate Systems 
Command. The Systems Commands then transmit the reviews 
to NMQAO. NAVSEA activities submit copies directly to 
NMQAO. [Ref. 7] 

10. Certificates of Competency (COCs) . If a small business 
is determined to be non-responsible by a Government 
buying activity, the small business can request that the 
Small Business Administration (SBA) determine whether 
the business is responsible. If the SBA concludes that 
the small business is responsible, it will prepare a COC 
to document that determination. The buying activity 
must then treat the small business as a responsible 
offeror. [Ref. 3:p. 1-8] The SBA sends hard copies of 
COCs to NMQAO for inclusion in PDREP [Ref. 7]. 

The CES data base excludes: 

1. Material evaluations for base application and local use. 

2. Contractors developing major weapon systems. 

3. Medical procurements, material, suppliers, or 
evaluations . 

4. Subsistence procurements, material, suppliers, or 
evaluations . 

5. Unsatisfactory material condition caused by improper 
handling after receipt, deterioration during local 
storage, or inadequate maintenance or operation. 
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6. Transportation discrepancies caused by the carrier. 

7. Ammunition and explosives accidents. 

8. Nuclear weapon procurements, material, suppliers, or 
evaluations . 

9. Naval Nuclear Power Plant primary system procurements, 
material, suppliers, or evaluations. 

10. Strategic Systems Project Office procurements, 

. suppliers, or material evaluations. [Ref. 3:p. 1-3] 

C. RED YELLOW GREEN PROCEDURES 

A one year test of the RYG program commenced 1 August 
1989. It is being conducted at the following five Navy 
activities: Naval Air Engineering Center Lakehurst, Naval 

Avionics Center Indianapolis, Naval Ships Parts Control Center 
(SPCC) Mechanicsburg (Code 021, Level 1/SS) , Naval Supply 
Center Charleston/Naval Shipyard Charleston, and Naval Supply 
Center Pensacola/Naval Aviation Depot Pensacola. [Ref. 2] 

The RYG program is a source selection tool that uses 
information documented in the CES to evaluate contractor past 
performance. Based on information in the CES, contractors are 
classified by FSC as high risk quality performers (Red) , 
moderate risk (Yellow) , or low risk (Green) . New contractors 
or those for which the CES does not have enough data are 
classified as "Insufficient Data." Because the classification 
is by FSC and contractor, one contractor may have several 
color classifications if he supplies the Government with more 
than one commodity. [Ref. lrencl. (1), p. 3] 
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The general descriptions of the color classifications as 
outlined in the program procedures are: 

1. Red. The contractor's performance history indicates he 
has supplied goods or services in a particular FSC of 
poor enough quality to warrant requirement of special 
quality actions and higher level review before contract 
award. He is designated as a high quality risk. 

2. Yellow. The contractor's performance history indicates 
he has supplied goods or services in a particular FSC of 
poor enough quality to warrant requirement of special 
quality actions. He is designated as a moderate quality 
risk. 

3. Green. The contractor's performance history indicates 

that he has supplied high quality goods or services and, 
therefore, is designated as a low quality risk. His 
proposal is evaluated without anticipating special 
quality actions. [Ref. 1 : end . (1), pp. 3-4] 

Contractors are classified as Red, Yellow, or Green based on 

the specific criteria listed in Appendix A [Ref. lrencl. (1), 

attachment (2)]. The RYG data base will be updated monthly to 

reflect a particular contractor's current classification [Ref. 

1 : end . ( 1) , p . 5 ] . 

If no negative quality performance data are received on 
Red or Yellow contractors during the previous 12 months, they 
will be reclassified. Red contractors will automatically be 
reclassified as Yellow, and Yellow contractors will be placed 
in the Insufficient Data classification. If a contractor was 
classified as Red or Yellow because of his reject rate on 
MIRs, and if no rejects have occurred on the inspection of 
five or more lots of material during the past six months, he 
will be reclassified as Yellow or Green respectively. If a 
contracting officer has reason to question a commodity 
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classification, NMQAO should be contacted for confirmation of 

the current classification. [Ref. l:encl. (1), p. 5] 

Contractor classifications will be updated by NMQAO 

monthly based on new data received. After each update, NMQAO 

will send a computer tape to each of the five test activities 

reflecting the new contractor classifications. The most 

current data will then be used by the activities in 

performance of the test. [Ref. 12] 

The RYG test procedures stress that a contractor's color 

classification should never keep him from competing for a 

Government contract. They specifically state: 

The procedures set forth herein do not in any way obliterate 
the requirement that a written responsibility determination 
is to be made for every requirement prior to award. A 
contractor's color classification alone is not sufficient to 
determine that a contractor is or is not responsible without 
further consideration of the standards set forth in FAR 
9.104. [Ref. 1 : end . (1), p. 4] 

The detailed procedures of the RYG program are divided 

into three categories, simplified small purchase, major 

purchase, and fixed price-greatest value procedures. A 

description of the application and specific requirements of 

each of these categories follows. 

1 . Simplified Small Purchase Procedures 

The simplified small purchase procedures will be 

tested at all five test activities. They apply to all oral or 

written quotations solicited during the one year test for 

selected FSCs that will result in purchase orders with an 

estimated value greater than $2500. The FSCs used for the 
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test were chosen by each test activity. They may be modified 
during the test by agreement between the test activity and 
NMQAO. Each activity can choose whether to employ these 
procedures for purchases under blanket purchase agreements, 
imprest fund purchases, and delivery orders against 
established contracts or with the General Services 
Administration. However, if RYG procedures are used for these 
purchases, the activity must issue a written or confirming 
purchase order to document the purchase for the file. Waiver 
of the use of these procedures may be granted by the Chief of 
the Contracting Office for all test sites except SPCC. At 
SPCC, the Director of the Hull, Mechanical, and Electrical 
Contracting Department must grant any waivers. [Ref. l:encl. 

(1) , pp. 1-2] 

When a purchase made under these procedures requires 
a synopsis in the Commerce Business Daily, the notice will 
explain that, although price will be a significant factor in 
proposal evaluation, other factors, including the contractor's 
quality history, will also influence the award decision. 
Likewise, the solicitation document must include a clause 
explaining the RYG program to prospective offerors and 
advising them of its use for the solicited procurement. [Ref. 
l:encl. (1), p. 6] The clauses to be used for simplified 
small purchase procedures are provided in Appendix B [Ref. 
l:encl. (1), attachment (6)]. 
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Since the basis of a contractor's color classification 



by FSC does not vary, the level of quality risk is the same 
regardless of the procedures used for the procurement (i.e., 
small purchase, major procurement, or greatest value) . 
However, the procedures that contracting personnel follow 
after receiving proposals does vary. Under the simplified 
small purchase procedures, the following procedures are 
followed . 

First, each offeror's classification must be 
determined from the data base — Red, Yellow, Green, or 
Insufficient Data. If the contractor is classified Green, his 
proposal is evaluated as received since past performance 
indicates that quality should not be a concern if he is 
selected. If the offeror is classified Yellow, the 
appropriate TEA is added to his quoted price. The adjusted 
price should better represent what the actual cost to the 
Government would be (considering the cost to correct quality 
problems) if he were selected. If the contractor is 
classified Red, again his quoted price is adjusted by adding 
an appropriate TEA. This TEA will be higher than the one 
assigned to a Yellow contractor since the costs the Government 
incurs while doing business with a Red contractor will 
probably be greater than that of a contractor classified as 
Yellow. If there are insufficient data to assign the 
contractor a color classification for the FSC involved in the 
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solicitation, there is no basis for adding a TEA to his cost 
proposal, so none is assigned. [Ref. l:encl. (1), pp. 3-4] 

After TEAs are assigned to proposals from Red and 
Yellow contractors, adjusted prices are used in the source 
selection process. If a contract is awarded to other than a 
Green contractor, additional actions must be taken by the 
purchasing activity during the test period. It is anticipated 
that similar actions will be required when RYG is exported for 
general use by procuring activities. If award is made to a 
Red contractor, for example, the head of the contracting 
office must approve the award. The RYG test procedures 
stress, however, that under no circumstances is a Red 
contractor to be barred from competing for a contract. [Ref. 
lrencl. (1) , p. 4] If award is made to either a Red or Yellow 
contractor, contracting personnel should ensure that any 
necessary quality control requirements are included in the 
purchase order [Ref. l:encl. (1), p. 6]. If award is made to 
a contractor on which the Government has insufficient quality 
data, even though no TEAs were assigned, the procuring 
activity may choose to require greater quality controls than 
they would if dealing with a Green contractor [Ref. l:encl. 
(1) , p. 4] . 

The TEAs for the simplified small purchase procedures 
are set at standard values [Ref. l:encl. (1), p. 2]. The TEA 
for a Red contractor is $2499 and for a Yellow contractor is 
$1255 [Ref. 1 : end . (1), attachment (4)]. These values are 
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based on the anticipated requirement for the following 
additional Government quality actions. For a Red contractor, 
additional requirements would be Government source inspection 
(GSI ) , receipt inspection at destination performed with a Navy 
representative present, and a quality assurance letter of 
instruction. For a Yellow contractor, the requirements would 
be GSI and a quality assurance letter of instruction. [Ref. 
l:encl. (1), attachment (3)] The dollar values were 
calculated by estimating the costs the Government would incur 
in performing these activities. Specifically, GSI is assigned 
a cost of $500 by estimating this would take the Government 14 
hours at $34.18 per hour. Receipt inspection at destination 
is expected to cost the Government $1194 based on eight hours 
at $43 per hour plus $200 for material handling and $650 for 
testing. Finally, the quality assurance letter of instruction 
is estimated to cost $755 based on eight hours at $34.18 per 
hour for DCAS personnel plus 16 hours at $30 per hour for 
procurement representative personnel. [Ref. 1 : end . (2), 
attachment (3) ] 

2 . Major Purchase Procedures with TEAs 

The major purchase procedures with TEAs will be used 
at only two of the five test activities, SPCC Mechanicsburg 
(Code 021, Level I/SS) and Naval Supply Center Pensacola/Naval 
Aviation Depot Pensacola. These procedures apply to all 
negotiated competitive solicitations with an estimated value 
greater than $2500 for the FSCs covered by the test during the 
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one year test period when large purchase procedures are used. 
As with the simplified small purchase procedures, each test 
activity has chosen the FSCs to be analyzed during the test, 
and these may be modified by agreement between the test site 
and NMQAO. Waiver of the procedures may be granted by the 
Chief of the Contracting Office or, at SPCC, by the Director 
of the Hull, Mechanical, and Electrical Contracting 
Department. [Ref. l:encl. (2), pp. 2-3] 

When a procurement covered by these test procedures 
requires a synopsis in the Commerce Business Daily, the notice 
will explain that, although price will be a significant factor 
in proposal evaluation, other factors as detailed in the 
solicitation, including the offeror's quality history, will 
influence the award decision [Ref. lrencl. (2), p. 7]. The 
solicitation will include essentially the following notice to 
offerors : 

This procurement is part of a test of the Navy's Contractor 
Evaluation System, "Red/Yellow/Green" Program. Award will 
be based upon the Contracting Officer's decision as to which 
offer provides the best value to the Navy--price, past 
quality performance, and other factors considered. Details 
are provided in the provisions entitled "NOTICE TO 
PROSPECTIVE OFFERORS" (Section L) and "ADDITIONAL EVALUATION 
FACTORS — TEST OF CONTRACTOR EVALUATION SYSTEM (MAJOR 
PURCHASE PROCEDURES) " (Section M) . [Ref. lrencl. (2), p. 7] 

The clauses to be included in Sections L and M of the 

solicitation are provided in Appendix C [Ref. lrencl. (2), 

attachment (5)]. 

When proposals are received, contracting personnel 
query the RYG data base to determine the color classification 
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of each offeror for all FSCs covered by the solicitation. As 
with simplified small purchase procedures, a Green 
classification indicates that the offeror is a low quality 
risk and, therefore, no TEAs are assigned to his proposal. If 
the contractor has a Yellow classification, the procurement 
package and the contractor's quality history will be used to 
determine necessary Government quality assurance actions and 
associated TEAs. The TEAs are then added to the offeror's 
proposed price. If the contractor has a Red classification, 
appropriate TEAs are determined in a manner similar to those 
for Yellow contractors. However, a greater number of quality 
assurance actions will be expected when dealing with a Red 
contractor. Therefore, the TEAs assigned will probably be 
higher. If the contractor does not have a color 
classification because of insufficient data, he will not be 
assigned any TEAs. [Ref. l:encl. (2), p. 4] Additionally, if 
the procurement is from a sole source supplier, no TEAs will 
be assessed regardless of the applicable color classification 
[Ref. l:encl. (2), p. 6]. 

The TEAs for major purchase procedures are based on 
the specific quality deficiencies in the performance history 
of the contractor for a particular FSC. These deficiencies 
are used to determine which additional quality assurance 
actions the Government should take to ensure receipt of a 
quality good or service. [Ref. 1 : end . (2), p. 2] 
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Contracting personnel take the following steps when assigning 
TEAs under major purchase procedures. 

First, contracting officers ascertain the reason for 
the contractor's classification being Red or Yellow by pulling 
his classification code from the RYG data base (listed in 
Appendix A) . Second, they decide which additional quality 
assurance actions the Government must take. A guideline is 
provided to assist them (Appendix D) [Ref. ltencl. (2), 
attachments (3)&(4)]. As can be seen in Appendix D, the 
contractor's deficiency code or codes are tied to an alpha- 
numeric code for additional quality assurance requirements. 
Now, contracting personnel perform the final step, assignment 
of TEAs. The alpha-numeric codes for quality assurance 
actions correspond to the list of quality assurance 
requirements and associated TEAs in Appendix D. The TEAs 
listed are calculated in the same way as those used in the 
simplified small purchase procedures. Specific calculations 
for each TEA are detailed in Appendix D. The test procedures 
stress, however, that the costs on which the listed TEAs are 
based are provided as examples. Since labor rates and test 
costs may vary considerably between procuring activities, each 
test activity should recalculate the TEAs to reflect the 
procuring activity's estimated costs. 

After TEAs are assigned to appropriate proposals, all 
proposals are evaluated together (using adjusted prices for 
contractor's assigned a TEA) [Ref. l:encl. (2), p. 6]. If the 
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contract is awarded to a Red or Yellow contractor, the 
contracting officer must ensure that the additional quality 
assurance actions used to calculate TEAs are called out as 
requirements in the contract [Ref. ltencl. (2), p. 7]. If the 
contract is awarded to a Red offeror, approval for the award 
must be granted by the head of the contracting office [Ref. 
l:encl. (2), p. 4]. If a contractor on which the Government 
has insufficient quality data wins the award, the procuring 
activity will require appropriate quality assurance actions to 
ensure a certain quality level even though TEAs were not 
assigned [Ref. l:encl. (2), p. 4]. 

3 . Major Purchase Procedures with Fixed Price-Greatest 

Value 

The greatest value procedures will be used at only 
three of the five test activities; Naval Air and Engineering 
Center Lakehurst, Naval Avionics Center Indianapolis, and 
Naval Supply Center Charleston/Naval Shipyard Charleston [Ref. 
l:encl. (3), p. 2]. These procedures apply to all negotiated 
competitive solicitations with an estimated value greater than 
$2500 for the FSCs covered by the test during the one year 
test period where sealed bidding procedures would otherwise be 
most appropriate [Ref. ltencl. (3), p. 2]. Each test activity 
has chosen the FSCs that will be covered by the test; these 
may be modified by agreement between the activity and NMQAO 
[Ref. ltencl. (3), p. 3]. Waiver of the procedures may be 
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granted by the Chief of the Contracting Office [Ref. l:encl. 
(3) , p. 3] . 

Solicitations prepared under these procedures will 
include the same notice used under the major purchase 
procedures employing TEAS [Ref. l:encl. (3), p. 7], The 
clauses to be included in Sections L and M of the solicitation 
are provided in Appendix E [Ref. l:encl. (3), attachment (3)]. 
The source selection/evaluation plan will be prepared by the 
Quality staff and will specify the criteria to be used in 
evaluating Red, Yellow, and Green commodities. This plan will 
be forwarded with the procurement request to the contracting 
officer for review and approval. [Ref. l:encl. (3), p. 3] 
For those commodities covered by these procedures, the source 
selection/evaluation plan will assign price a minimum weight 
of 40 percent, with the remainder assigned to quality 
assurance considerations under RYG procedures [Ref. l'.encl. 
(3), p. 2]. During the test, no factors other than these two 
shall be considered without prior approval from Naval Supply 
Systems Command (NAVSUP) Code 02 via NAVSUP Code 021 [Ref. 
lrencl. (3), p. 2]. Appendix F [Ref. l:encl. (3), p. 8] 
provides an example of the RYG evaluation portion of the 
source selection/ evaluation method. 

If a contractor is determined to be Green, he will 
normally be rated as "Excellent" or "Good." A Yellow 
classification will normally carry with it a rating of "Good," 
"Average," or "Marginal." Red commodities will usually be 
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rated "Marginal" or "Unacceptable." If there are insufficient 
data to evaluate the offeror, the proposal will be considered 
technically acceptable and will be evaluated solely on the 
basis of price. The contracting officer can exercise 
discretion in the use of adjectives (i.e., excellent, good, 
etc.) for each color classification or other locally 
determined source selection criteria. [Ref. l:encl. (3), pp. 
4-5] 

When proposals are received, the contracting office 
will forward the names and Commercial and Government Entity 
(CAGE) codes of all offerors to the Quality staff. The 
Quality staff will determine the color classification and the 
associated reason codes for each offeror for each FSC. They 
will then assign ratings to each offeror based on the criteria 
outlined in the source selection plan. If commodities of 
different offerors lie within the same color classification, 
their relative differences will be explained in an 
accompanying narrative. This evaluation will be forwarded to 
the contracting office. [Ref. l:encl. (3), p. 3] 

If the contract is awarded to a Red or Yellow 
contractor, the Quality staff will determine any necessary 
additional quality assurance requirements that should be 
imposed on the contractor. The contracting officer will 
ensure that these are included in the contract. [Ref. l:encl. 
(3), p. 3] If the contract is awarded to a Red contractor, 
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approval for the award must be granted by the head of the 
contracting office [Ref. l:encl. (3), p. 3]. 

D. SUMMARY 

This chapter described the Navy's RYG program. First, it 
explained reasons behind establishment of the program. 
Second, it described the CES data base on which the program is 
based. Finally, it detailed the test procedures that are 
being used to implement the program. The next chapter will 
briefly describe three other current DOD contractor quality 
programs. 
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Ill . OTHER DEPARTMENT OF DEFENSE QUALITY INITIATIVES 



A. INTRODUCTION 

One factor in the researcher's evaluation of the Red 
Yellow Green (RYG) program is an assessment of how this 
program compares with other contractor quality programs being 
used or considered for use by the Navy. Three programs 
evaluated by the researcher are the Air Force Contractor 
Performance Assessment Reporting System (CPARS) , the Naval 
Avionics Center (NAC) Indianapolis Blue Ribbon Contractor 
program, and the Department of Defense (DOD) Total Quality 
Management (TQM) initiative. These three programs were chosen 
for the assessment because each has been considered for use 
by the Navy as an alternative to RYG. Therefore, a discussion 
of the merits and shortcomings of these programs and their 
relationship to RYG is considered important. The evaluation 
described above will be the subject of Chapter IV of this 
thesis. In preparation for the assessment in Chapter IV, this 
chapter provides background information on the procedures 
associated with each of the three programs. 

B. CONTRACTOR PERFORMANCE ASSESSMENT REPORTING SYSTEM 

The first program evaluated is CPARS used by the Air 
Force. As stated in Air Force Systems Command (AFSC) 
Regulation 800-54, "The sole purpose of CPARS is to provide 
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for a command-wide performance data base used in AFSC source 
selections." [Ref. 13] CPARS currently includes 84 

contractors and their divisions and subsidiaries. These 
contractors were chosen because the Air Force conducts most of 
its business with them. Therefore, it was determined that the 
benefits of maintaining performance data on them would be 
worth the effort involved in the CPARS process. [Ref. 14] A 
Contractor Performance Assessment Report (CPAR) is prepared 
on contracts with the 84 contractors included in the program 
valued at more than $5 million. Preparation of the reports 
are further limited to contracts for concept demonstration and 
validation, full-scale development, and full-rate production 
and deployment. The program excludes contracts for laboratory 
work, services, and operations and maintenance. [Ref. 13 :p. 
1] Following are the areas of contractor performance 
evaluated in the CPAR: 

1. Product system performance. 

a. Engineering design/support; 

b. Software development. 

2. Schedule. 

3. Cost performance. 

4. Product assurance. 

5. Test and evaluation. 

6. Integrated logistics support program. 

7. Management responsiveness. 

8. Subcontract management. 
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9. Variances. 

a. Cost; 

b. Schedule; 

10. Other areas unique to the contract evaluated. [Ref. 15] 
The CPAR is prepared by the program director or program 
manager. An initial CPAR for a new contract is prepared 
between six months and one year after contract award. 
Following this, CPARs are submitted annually during the 
contract performance period. Additional CPARs are completed 
when the program manager changes or if the program manager 
determines that a contractor's performance has changed 
sufficiently to warrant modification of the most recent CPAR. 
In addition, the contractor can request that a CPAR be 
completed before the annual report is prepared if he believes 
that his performance has changed enough to alter the 
assessment. The final CPAR is prepared within six months 
after the final major end item is delivered or when a contract 
is terminated. [Ref. 13 :p. 2] 

Preparation of the CPAR involves four steps. First, the 
program manager or engineer responsible for the contract 
prepares a preliminary assessment with associated 
documentation. Second, the program manager forwards the 
original of the preliminary CPAR to the contractor and retains 
a copy. The contractor has 30 days to respond to the CPAR. 
Upon receiving a response from the contractor or 30 days after 
transmitting the CPAR to the contractor, whichever occurs 
first, the program manager takes the third step and signs the 
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CPAR . He may first revise his evaluation if the contractor's 
response indicates that a change is appropriate. Finally, the 
program manager sends the signed CPAR with its attachments to 
the Product Division of the AFSC having cognizance over his 
program for input into the command-wide data base. [Ref. 
13 : p . 2] 

As part of his assessment, the program manager translates 
his evaluation of the contractor in each of the ten 
performance evaluation areas into a color classification. The 
following is a description of the criteria he must use in 
assigning colors: 

Blue (Exceptional) . Indicates performance clearly 
exceeds contractual requirements. The area of evaluation 
contains few minor problems for which corrective actions 
appear highly effective. For cost performance, blue 
indicates a positive cost variance. 

Green (Satisfactory) . Indicates performance clearly 
meets contractual requirements. The area of evaluation 
contains some minor problems for which corrective actions 
appear satisfactory. For cost performance, green indicates 
no cost variance or a negative cost variance greater than 
zero but less than or equal to five percent. 

Yellow (Marginal) . Indicates performance meets 
contractual requirements. The area of evaluation contains 
a serious problem for which corrective actions have not yet 
been identified, appear only marginally effective, or have 
not been fully implemented. For cost performance, yellow 
indicates a negative cost variance greater than five percent 
but less than or equal to 15 percent. 

Red (Unsatisfactory) . Indicates the contractor is in 
danger of not being able to satisfy contractual requirements 
and recovery is not likely in a timely manner. The area of 
evaluation contains serious problems for which corrective 
actions appear ineffective. For cost performance, red 
indicates a negative cost variance greater than 15 
percent. [Ref. 13:p. 7] 
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Since each area is evaluated by color, a contractor might 
receive several color classifications under one contract. For 
example, he might be classified Red in quality, Green in cost 
control, and Blue in the remainder. 

When making assessments of contractors, the AFSC 
Regulation covering CPARS cautions program managers to base 
their evaluations on facts. Their assessments should be 
supported by contract management data "such as cost 
performance reports, technical interchange meetings, financial 
solvency assessments, production management reviews, 
contractor operations reviews, functional performance 
evaluations, and earned contract incentives." [Ref. 13 :p. 1] 
The Regulation specifically states, "Subjective assessments 
concerning the causes or ramifications of the contractor's 
performance should be provided; however, speculation or 
conjecture should not be included." [Ref. 13 :p. 1] 

The intention of the program is that during source 
evaluation, CPARs will be used to evaluate a contractor's past 
performance and assess the risk involved in doing business for 
the contract under consideration. Specifically, during the 
source selection process, the contracting officer will check 
the CPARs of contractors submitting proposals. If he 
determines that the effort described in the CPARs applies to 
the current procurement, they will be assembled into a report 
for the Performance Risk Analysis Group (PRAG) . The PRAG 
considers the CPARs along with other contractor past 
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performance data to determine contractor risk. The PRAG 
usually expresses contractor risk as low, medium, or high. 
This risk assessment is then used as part of the source 
selection. [Ref. 14] 

At present, each Product Division within the AFSC 
maintains a complete set of hard copy CPARs . The Air Force 
intends eventually to automate CPARS. The automation effort 
has been delayed for two reasons. First, the resources 
required to input the CPARs and ensure compatibility with the 
computer hardware at all commands are not currently available. 
Second, the Air Force is still developing a method to ensure 
the security of the automated data. [Ref. 14] 

Each CPAR will be retained for five years unless the 
program manager requests a longer retention period. CPARs are 
considered source selection sensitive because they will be 
used for procurements throughout their retention period. They 
may also contain proprietary information. Therefore, all 
CPARs and attachments will be marked "For Official Use Only/ 
Source Selection Sensitive." Additionally, the forms and the 
data base (if automated) must be protected from disclosure to 
organizations or personnel not involved in the source 
selection process. [Ref. 13:p. 3] 

C. BLUE RIBBON CONTRACTOR PROGRAM 

The Blue Ribbon Contractor program at NAC Indianapolis 
began in December 1987. It was inspired by the Packard 
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Commission recommendations of 1986 [Ref. 16] and the Blue 
Ribbon Contractor program at Tinker Air Force Base begun a 
year earlier [Ref. 17]. NAC has attempted to learn from some 
of the difficulties experienced at Tinker and has tailored the 
program to respond to their concerns and conform to their 
resource constraints [Ref. 18]. 

The program at NAC is applied to contracts, purchase 
orders, and blanket purchase agreements with up to a $25,000 
estimated value [Ref. 16:p. 41]. Procurements made for the 
bomb-rack program, which have historically caused difficulties 
for NAC, are excepted from this dollar limit and are all 
included in the Blue Ribbon Contractor program [Ref. 19]. 
Other exemptions to the program are procurements awarded via 
sealed bidding procedures and involving small, disadvantaged 
business set-aside awards. Additionally, the program is not 
a substitute for procedures governing contractor respon- 
sibility determination [Ref. 19:p. 2]. 

In order to qualify as a Blue Ribbon contractor, a 
contractor must first apply in writing to NAC. If he has made 
one delivery during the last 12 months, he will be placed on 
the list in that category. To receive the benefits of being 
designated a Blue Ribbon contractor, however, he must have and 
maintain an on-time delivery rate of 95 percent or greater. 
[Ref. 19 :p. 2] This delivery rate is calculated by dividing 
the number of delinquent orders by the number of open orders 
and multiplying by 100 [Ref. 18]. Only data collected at NAC 
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Indianapolis are used in determining this delivery rate [Ref. 
19:p. 3]. If a Blue Ribbon contractor does not qualify to 
receive the benefits of Blue Ribbon status upon application, 
or if his on-time delivery rate drops below 95 percent, he is 
notified that he is still listed as a Blue Ribbon contractor, 
but he cannot exercise the benefits of the program until his 
delivery rate improves [Ref. 18]. 

There are three benefits accruing to Blue Ribbon 
contractors with a qualifying delivery rate. First, the 
contracting officer has the authority to award to a Blue 
Ribbon contractor over other offerors at a price up to ten 
percent higher than the lowest priced responsible offeror. If 
the procurement is in support of the Navy bomb rack program, 
the contracting officer has the authority to award to a Blue 
Ribbon contractor at a price up to 20 percent higher than the 
lowest priced responsible offeror. [Ref. 19 :p. 1] Second, 
NAC Indianapolis solicits Blue Ribbon contractors to the 
maximum extent possible. The bidders list is rotated, but 70 
percent of it is made up of Blue Ribbon contractors. [Ref. 
18] Third, Blue Ribbon contractors are permitted to use their 
status in their marketing programs [Ref. 18] . 

So far, the Blue Ribbon Contractor program has resulted in 
only 3.5 percent of awards (of contracts in the program) being 
made to Blue Ribbon contractors that were not the lowest 
priced responsible offerors. However, the delinquency rate on 
blanket purchase agreements has dropped from 67 percent before 
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the start of the program to 8.7 percent in July 1989. Seventy 
percent of the suppliers awarded contracts in July 1989 are 
the same suppliers that were receiving awards when the program 
began. One protest has been filed by a low offerer who lost 
a contract to a higher priced Blue Ribbon contractor. The 
General Accounting Office dismissed the protest because it was 
not timely. [Ref. 18] Therefore, no protest related to the 
Blue Ribbon Contractor program has yet tested the program's 
criteria . 

D. TOTAL QUALITY MANAGEMENT 

TQM is a management style introduced to the Japanese in 
the late 1940s by Americans participating in the post-World 
War II effort to rebuild Japan. TQM proved to be a very 
successful management method in Japan. As a result, it is 
gaining an increasing following within U.S. industry trying 
to regain competitive position both internationally and 
domestically. Based on successes in Japan and within the 
U.S., DOD is attempting to adopt TQM in an effort to operate 
more efficiently in a time of dwindling resources. This 
section of Chapter III will attempt to describe the TQM effort 
as it applies to DOD and, more specifically, DOD procurement. 
First, it will describe TQM in general, concentrating the 
discussion on those aspects of TQM that most directly impact 
upon procurement practices and, therefore, that would compete 
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with RYG . Second, it will outline current DOD implementation 
plans related to acquisition. 

1 . TOM Implementation — General 

Adopting TQM means that everyone within an 

organization must be involved in improving the quality of 
their product. Whether they are manufacturing a product or 
preparing documents, everyone produces something and every 
product has a customer. Therefore, everyone in the 

organization can improve quality. The basic theory behind TQM 
is that as quality improves, productivity improves because 
rework is reduced [Ref. 20] . As quality and productivity 
rise, competitive position also improves [Ref. 21]. 

One of the key figures in the introduction of TQM to 
the Japanese is Dr. W. Edwards Deming. Dr. Deming has devised 
14 principles for management which he believes U.S. firms must 
follow if they desire to adopt TQM. His 14 principles are: 

1. Create constancy of purpose toward improvement of 
product and service, with the aim to become competitive 
and stay in business, and to provide jobs. 

2. Adopt the new philosophy. We are in a new economic age. 
Western management must awaken to the challenge, must 
learn their responsibilities, and take on leadership for 
change . 

3. Cease dependence on mass inspection to achieve quality. 
Eliminate the need for inspection on a mass basis by 
building quality into the product in the first place. 

4. End the practice of awarding business on the basis of 
price tag. Instead, minimize total cost. Move toward 
a single supplier for any one item, on a long-term 
relationship of loyalty and trust. 
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5. Improve constantly and forever the system of production 
and service, to improve quality and productivity, and 
thus constantly decrease costs. 

6. Institute training on the job. 

7. Institute leadership (see Point 12). The aim of super- 
vision should be to help people and machines and gadgets 
to do a better job. Supervision of management is in 
need of overhaul, as well as supervision of production 
workers . 

8. Drive out fear, so that everyone may work effectively 
for the company. 

9. Break down barriers between departments. People in 
research, design, sales, and production must work as a 
team, to foresee problems of production and in use that 
may be encountered with the product or service. 

10. Eliminate slogans, exhortations, and targets for the 
work force asking for zero defects and new levels of 
productivity. Such exhortations only create adversarial 
relationships, as the bulk of the causes of low quality 
and low productivity belong to the system and thus lie 
beyond the power of the work force. 

11. Eliminate work standards (quotas) on the factory floor. 

Substitute leadership. Eliminate management by 

objective. Eliminate management by numbers, numerical 
goals. Substitute leadership. 

12. Remove barriers that rob the hourly worker of his right 
to pride of workmanship. The responsibility of super 
visors must be changed from sheer numbers to quality. 
Remove barriers that rob people in management and in 
engineering of their right to pride of workmanship. 
This means, inter alia, abolishment of the annual or 
merit rating and of management by objective. 

13. Institute a vigorous program of education and self- 
improvement . 

14. Put everybody in the company to work to accomplish the 
transformation. The transformation is everybody's job. 
[Ref. 2 0 : pp . 23-24] 

Although all 14 points are interrelated, two have the 
greatest implications for changing traditional procurement 
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practices. These are number 3, "Cease dependence on mass 
inspection" and number 4, "End the practice of awarding 
business on the basis of price tag." Therefore, the 
discussion in this section will focus on how these two 
principles should be implemented. 

The push to do away with mass inspection is based on 
the theory that inspection is ineffective in ensuring quality. 
This is true for two reasons. First, inspection occurs too 
late in the process. If quality has not been built in during 
production, it cannot be inspected in [Ref. 20:pp. 28-29]. 
Inspection can only separate good items from bad. Second, 
even if 100 percent inspection is employed to separate good 
from bad, some defective material will slip through. One 
hundred percent inspection is unreliable because inspectors 
are human and make errors, no matter how conscientious, 
because of boredom and fatigue. [Ref. 20 :p. 29] In addition 
to being ineffective, mass inspection is very expensive. 
First, it is very labor intensive, so it requires the 
employment of many inspectors. Second, the items that fail 
inspection are either discarded or reworked. The first 
alternative wastes the costs of material and production used 
to make the item. The second alternative wastes the 
additional material and man hours employed in reworking the 
item. These are all resources that could be better employed 
elsewhere . 
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In place of mass inspection, Statistical Process 
Control (SPC) should be employed. SPC is a method to build 
quality into a product during production. It is a tool to be 
used by management to identify the reasons a product is not 
produced with consistency. Management charts a process, 
perhaps one segment of production, to see the fluctuations in 
the output of the process. If the process exhibits more than 
simple random variation, it is not in statistical control. 
This means that local sources of trouble are affecting the 
process. [Ref. 21:p. 5] Examples of local problems are 
differences in materials used in production, readjustment of 
machinery at the beginning of each shift, and use of different 
procedures on each shift. These local problems must be 
eliminated to bring the process into statistical control. 
Only when the process is in control, can improvements be made 
effectively. This is because if the process is not in 
control, management will not be able to observe the affects of 
changes made to it. [Ref. 21:p. 5] Once the process is in 
control, changes made to it will shift the entire control 
chart in either a favorable or unfavorable direction, giving 
management feedback about its innovations. Management can now 
follow the Plan, Do, Check, Act (PDCA) cycle. First, data are 
analyzed to determine what change might improve the process. 
Second, management tests the idea by putting it into practice. 
Third, the control charts are monitored to determine the 
affect (favorable or unfavorable) of the change. Finally, 
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based on the feedback from the third step, the change is 
either implemented permanently or discarded. The PDCA cycle 
is then begun again. The result is continuous improvement of 
the process in smaller and smaller increments. 

The key to employing sampling inspections and SPC in 
place of mass inspection is the determination of which 
characteristics to monitor and which inspection method to use. 
Dr. Ishikawa explains that it is important to inspect quality 
characteristics that are unstable, thus causing defects, as 
well as vital characteristics that affect the life of a 
product [Ref. 22]. These inspections are not performed after 
the product is complete; they are conducted by the worker 
during the production process. In addition, only a sample of 
the items produced are inspected to either determine whether 
the process is in control or to be sure that the process 
remains in control. Because only a small sample of items are 
inspected, inspectors can do the job properly. [Ref. 20:pp. 
29-30] Dr. Ishikawa further explains that the goal of SPC 
should be a product with 100 per cent acceptable quality. SPC 
makes this goal feasible. [Ref. 22:pp. 117-118] Mass 
inspection, on the other hand, is guaranteed to result in some 
percentage of failures. 

The second principle that applies directly to 
procurement is number 4, "End the practice of awarding 
business on the basis of price tag." What this principle 
means is that when purchasing material, an organization should 
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consider the lowest total cost of the item rather than the 



lowest initial cost [Ref. 23]. The organization must have 

long term goals. Buyers have a new job. They have 

traditionally sought lower and lower prices for material with 

little or no concern for quality or service. [Ref. 21:p. 25] 

It is difficult to select a contractor with a higher initial 

price since it can be very subjective when defending the 

selection during an award protest. Now buyers must 

concentrate on quality because, as Dr. Deming states: 

Price has no meaning without a measure of the quality being 
purchased. Without adequate measures of quality, business 
drifts to the lowest bidder, low quality and high cost being 
the inevitable result. American industry and the U.S. 
Government, civil and military, are being rooked by rules 
that award business to the lowest bidder. [Ref. 23 :p. 23] 

In order to ensure receipt of quality goods and 
services, an organization must do two things. First, it must 
require that its suppliers practice TQM. Second, it must 
reduce the number of suppliers it deals with and seek long- 
term relationships with these suppliers. 

To verify that a supplier practices TQM, an 
organization should require the supplier to provide 
statistical evidence of quality. This means supplying the 
buying organization with control charts and proof that the 
supplier is working on changing its management style to 
conform with Deming 1 s principles. [Ref. 21:p. 113] The 

buying organization must be sure that the control charts 
provided represent the proper inspection technique taken at 
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the critical points in the production process using the 
correct sample size. This requires a close relationship 
between the buying activity and the company supplying the 
material. The activity buying material must be sure that the 
supplier is capable of producing a quality product before the 
item is produced. In Dr. Deming's words, "The customer that 
waits for delivery of material to learn what he has bought 
will take what he gets." [Ref. 20:pp. 40-41] This concept 
opens a new world of responsibilities to the contract 
administration activities within DOD. It suggests that their 
traditional role inspecting the quality of material after it 
has been produced must change. Instead, they would be 
expected to participate with the contractor in verifying the 
quality of his production processes before and during 
production of required material. 

The concentration on long-term goals and close buyer- 
supplier relationship requires a long-term relationship with 
a small number of suppliers. This is in direct conflict with 
the Competition in Contracting Act of 1984 and a concept that 
DOD must deal with to successfully implement a TQM philosophy. 
The buying activity's goal is still to purchase material of 
high quality at the least cost. However, the buying activity 
must also devote resources to assist the vendor in employing 
TQM if it is to succeed. Therefore, as the number of 
suppliers an activity deals with increases, the number of 
resources devoted to ensuring a TQM philosophy also increases. 
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Even after the activity is working with suppliers dedicated to 
TQM and is receiving high quality material, if this material 
is supplied by two different vendors, there will be 
differences. These differences will surface when the material 
is placed into the production process. [Ref. 20:p. 35] For 
these reasons, Dr. Deming encourages organizations to reduce 
their suppliers for a given end item to one [Ref. 20, p. 23]. 
Ishikawa believes, however, that an organization should 
maintain at least two suppliers for a given product in case 
one firm is unable to supply the item [Ref. 24]. 

A supplier needs the promise of a long-term contract 
to make it cost effective for him to institute TQM and 
concentrate on continuously improving his production process 
[Ref. 21:p. 115]. Without some guarantee of continued 
business, he will not be willing to invest the resources 
necessary to bring his processes into control and then improve 
upon them. DOD must seriously consider the awarding of multi- 
year contracts (two to five years) or contracts with optional 
years to assure a contractor that DOD is supporting a basic 
TQM mandate. Contractors are currently forced to concentrate 
on making a profit in the short term since DOD is focused on 
the short term. If, however, a long-term relationship can be 
developed, both customer and supplier will benefit. The 
customer will receive high quality material at lower costs. 
The supplier will no longer need to employ resources in 
storing and reworking defective material. [Ref. 21 :p. 115] 
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2 . TOM Imolementat ion--DOD 



The application of TQM within the Federal Government 
began in 1986 when President Reagan signed Executive Order 
12552. The goal of this Executive Order is to significantly 
improve the productivity of the Federal Government by 1992 
through employment of TQM. [Ref. 25] In 1988, DOD initiated 
its TQM program, issuing the Total Quality Management Master 
Plan in August of that year. 

The strategy of the Master Plan "aims at achieving one 
broad, unending objective: continuous improvement of products 
and services." [Ref. 26] In order to achieve this objective, 
the plan includes goals for the short term (one year) , mid 
term (three years) , and long term (seven years) . These goals 
cover a broad range of DOD tasks. They state, however, that 
the acquisition community will be the first to implement TQM 
and that others will follow in approximately three years [Ref. 
26:p. 3]. Since the acquisition community is leading TQM 
implementation and this thesis will concentrate its discussion 
of TQM in the areas relating to RYG, discussion of the Master 
Plan will be limited to procurement and Government-industry 
relationships . 

In the short term, two of the seven goals directly 
impact procurement. The first of the two is to "Begin 
enlisting Defense industry commitment." This goal intends to 
begin TQM implementation by using programs already in 
existence within DOD. These programs are to be used to 
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communicate to industry DOD's desire for continuous process 
improvement. [Ref. 26:p. 10] The second applicable goal is 
to "Develop and implement recognition and reward system based 
on TQM goals and behaviors." This goal discusses the need to 
revise existing performance incentives to encourage and reward 
continuous improvement efforts. [Ref. 26 :p. 10] 

One of the nine mid-term goals applies directly to 
procurement: "Implementation commitment by major Defense 
contractors, with 'critical mass' achieved in at least the top 
25 contractors." This goal intends to result in a commitment 
from the top 25 Defense contractors to independently implement 
continuous process improvement programs. These 25 contractors 
receive contracts valuing more than 50 percent of DOD's total 
acquisition expenditures, and they receive most of DOD major 
system contracts. This commitment is expected to involve the 
training of senior leadership within these firms in TQM and 
their active involvement in continuous improvement efforts. 
[Ref. 16 : p . 6] 

In the long term, one of the four goals of the Master 
Plan impacts upon procurement: "Widespread Defense industry 
implementation of continuous process improvement." 
Implementation of this goal involves employment of TQM by all 
DOD prime and subcontractors. Its achievement requires three 
things of DOD. First, DOD must encourage and assist 
contractors in their implementation of continuous process 
improvement. Second, DOD is expected to develop criteria to 
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evaluate contractor commitment to continuous improvement. 
Finally, the degree and success of contractor process 
improvement efforts are to be included in source selection 
strategies. [Ref. 26:p. 4] Within this goal, the Master Plan 
states, "The intent of continuous process improvement efforts 
is to recognize and reward dedication and adherence to the 
improvement process, and not restrict reward to attaining a 
specific standard of performance." [Ref. 16:p. 4] 

E . SUMMARY 

This chapter briefly described three DOD contractor 
quality programs. The programs described were the Air Force 
CPARS , NAC Indianapolis Blue Ribbon Contractor program, and 
DOD TQM initiative. The procedures associated with each 
program have been provided as background information to 
support the next chapter. The chapter that follows will 
discuss the merits and shortcomings of the three programs and 
their relationship to RYG. 
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IV. A COMPARISON OF RED YELLOW GREEN AND OTHER 
CONTRACTOR QUALITY INITIATIVES 



A. INTRODUCTION 

This chapter will use the background information provided 
in Chapter III to analyze the Red Yellow Green (RYG) program 
relative to the other three programs described — the Contractor 
Performance Assessment Reporting System (CPARS) , the Blue 
Ribbon Contractor program, and the Total Quality Management 
(TQM) initiative. The purpose of the analysis is to determine 
whether RYG should co-exist with these other initiatives or if 
it should replace or be replaced by any one of them. 

The assessment will be organized in the same way as the 
background information presented in Chapter III. First, RYG 
will be compared to CPARS. Next, RYG will be evaluated 
relative to Naval Avionics Center (NAC) Indianapolis' Blue 
Ribbon Contractor program. Finally, RYG will be assessed for 
its compatibility with the Department of Defense (DOD) TQM 
initiative . 

B. CONTRACTOR PERFORMANCE ASSESSMENT REPORTING SYSTEM 

The CPARS devised by the Air Force is intended for use in 
source selections for major systems contracts. Therefore, the 
threshold for employment of CPARS is set at $5 million and the 
number of contractors included in the program has been limited 
to 84. The Air Force had two major reasons for setting these 
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limits. The first involves maintenance of the Contractor 
Performance Assessment Reports (CPARs) . The second involves 
the nature of the information reported in the CPARs and, 
therefore, the way in which this information must be used in 
the source selection process. These two factors will be 
discussed in detail in the following paragraphs. 

Maintenance of CPARs requires many resources. First, it 
involves the time invested by the program manager to prepare 
the report. Although the reports are supposed to be based on 
facts, they are not merely collections of data. They are 
written, subjective assessments of the contractor's perfor- 
mance and, therefore, require some effort on the part of the 
program manager. The second factor concerning maintenance of 
CPARs involves the efforts of those who verify the information 
contained in the reports, ensure that the reports are current, 
and physically store the reports. Since the CPARs are 
expected to be used in future source selections, the Air Force 
carefully verifies the information contained in the reports to 
ensure that it is compatible with past trends and other CPARs 
filed on the same contractor. When an activity uses CPARs in 
source selection, the currency of the information is checked. 
If a more current CPAR is desired, the procuring activity 
contacts the program office and requests more up-to-date 
information. In addition, since the system is not yet 
automated, maintaining hard copy files of all CPARs at the six 
Product Divisions within the Air Force Systems Command 
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requires a tremendous clerical effort. Maintenance of the 
CPARs is a very labor intensive process. 

The information contained in CPARs is subjective. It is 
based on facts, but much of it is still the opinion of the 
program manager and members of the program office. 
Additionally, because a CPAR is a written report, it is a 
relatively long document. Because of its subjectivity and 
length, the use of CPARs in source selection requires careful 
evaluation of the information contained in the report. To 
perform this evaluation, the CPARs must first be collected, 
evaluated for their relevance to the procurement in question, 
and assembled into an integrated report. The resulting report 
is then further evaluated in conjunction with other past 
performance data to determine a contractor's level of risk. 
This is a relatively long process. The Air Force believes the 
process is beneficial for major weapon system procurements. 
However, CPARS is not used for non-major acquisitions. 

The points discussed above suggest that CPARS would not be 
appropriate for contracting operations at field activities for 
three reasons. 

First, the investment in resources to maintain the CPARs 
could not be justified by the relatively small dollar 
procurements awarded by field contracting activities. The 
return on investment to obtain higher quality material would 
surely be less than the cost to prepare and maintain the 
CPARs . 
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Second, in order to be effective in field contracting, the 
number of contractors included in the program would have to be 
greatly expanded. The contractors that compete in major 
system procurements are relatively few. However, field 
contracting offices deal with thousands of contractors. Even 
if fewer than 100 contractors were included in the system, 
field activities would find it very difficult to obtain 
accurate past performance information. An expansion of CPARS 
at the field level to accommodate such a large number of 
contractors would be an impossible undertaking. 

Finally, even if CPARS were expanded, field contracting 
offices would have neither the time nor the resources to 
perform the evaluation necessary to correctly apply it to 
source selection. 

In summary, CPARS seems to be only appropriate for use in 
the procurement of major systems. It should not be considered 
for expanded use in field contracting. RYG was designed for 
use by field contracting offices. Accordingly, its formula 
driven assessment of past performance would not be appropriate 
for use in major system procurements. The two programs were 
designed to satisfy different needs in different areas of 
Government contracting. Therefore, neither should be 
considered as a replacement for the other. 
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C. BLUE RIBBON CONTRACTOR PROGRAM 

The Blue Ribbon Contractor program at NAC Indianapolis has 
five unique features. First, it is an automated system 
created by NAC using data collected by NAC. Second, the 
criteria used to define a Blue Ribbon contractor and the 
programs included in the program were chosen to focus 
attention on the critical and problem materials procured by 
NAC. Third, contractors must apply for Blue Ribbon status to 
be included in the program, and NAC performs the analysis to 
determine whether these contractors qualify for the benefits 
of the program. Fourth, the program concentrates on 
commodities important to NAC's mission, and a contractor must 
make a delivery to NAC at least once each year to be included 
in the program. Therefore, the contractors designated as Blue 
Ribbon will generally be the contractors NAC deals with on a 
regular basis. Fifth, use of the program in source selection 
requires judgment. Unlike RYG , it is not a formula-driven 
program. The decision to award to a Blue Ribbon contractor 
over a lower-priced offeror (not included in the program) 
requires an assessment by both contracting and quality 
assurance personnel. Therefore, the two groups must 
communicate with each other to use the program effectively. 

Like RYG, the Blue Ribbon Contractor program is ap- 
propriate for use in field contracting. However, a Blue 
Ribbon program requires that an activity devote many more 
resources than does RYG. First, the activity would need the 
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capability to automate. Second, it would have to devise the 
criteria to be used in designating Blue Ribbon contractors. 
NAC chose delivery, but other commands might be more 
interested in quality or reliability. Third, it would have to 
decide which commodities are important enough to its mission 
to justify inclusion in the program. Fourth, it would need to 
devote resources to the program to evaluate applicants and 
maintain the Blue Ribbon Contractor list. Finally, the 
contracting and quality assurance personnel in the command 
would have to be willing and able to work together to 
implement the program. 

Some commands would be unable to take the five steps 
discussed in the previous paragraph. They might lack the 
automation necessary to implement the program. Or, 
contracting personnel might be physically located away from 
quality assurance personnel. This would be the case for a 
Navy Regional Contracting Center procuring goods and services 
for many of its Navy field activities. In either case, 
implementation of a Blue Ribbon Contractor program would be 
impractical. Therefore, a Blue Ribbon program does not seem 
to be a viable option for every field contracting activity. 

The question then is whether RYG and Blue Ribbon 
Contractor programs could co-exist at activities capable of 
implementing both programs. This researcher believes the 
answer to that question is yes — the programs are compatible 
and can enhance each other. If used together, the Blue Ribbon 
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program would be used when procuring goods and services that 
the command considers critical to the performance of its 
mission. The Blue Ribbon program would allow the command to 
concentrate its attention on those parameters that it 
considers important. RYG could be used for all other procure- 
ments. By definition, RYG procurements would be for less 
critical and less frequently purchased materials. These items 
would not justify the resources needed to apply the Blue 
Ribbon program. In addition, the command may not have been 
able to generate enough local data on critical items to assess 
contractor past performance. RYG would allow the command to 
consider past performance in source selection while employing 
a very simple system. 

Selected field activities could benefit from using both 
RYG and a tailored Blue Ribbon Contractor program. The most 
likely evolution would be to first implement RYG since it is 
less complex. Use of RYG should help personnel involved in 
the source selection process become comfortable with using an 
automated system. In addition, implementation of RYG should 
force contracting and quality assurance personnel to work 
together. RYG requires quality assurance personnel to use 
past performance to evaluate offerors and determine the 
quality assurance actions required of each. They must then 
provide this assessment to contracting. Therefore, RYG 
involves quality assurance more deeply in the source 
evaluation process. Both will help pave the way for 
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introduction of a Blue Ribbon Contractor program concentrating 
attention on the area of the command's specialization. [Ref. 
27] 



D. TOTAL QUALITY MANAGEMENT 

The discussion in this section will focus on whether RYG 
furthers the goals of TQM. The purpose of this discussion is 
to evaluate whether RYG is consistent with DOD's TQM 
initiative or whether RYG hinders this initiative. This 
section will first consider RYG's conformance to the general 
goals of TQM as outlined in Chapter III. Second, it will 
relate RYG to the goals in the DOD TQM Master Plan. 

1 . TOM and RYG — General 

Two of Dr. Deming ' s 14 principles were determined to 
directly impact the Government procurement process and, 
therefore, apply to the comparison of TQM and RYG. These two 
principles are: 

1. Number 3. "Cease dependence on mass inspection to 
achieve quality. Eliminate the need for inspection on 
a mass basis by building quality into the product in the 
first place." [Ref. 20:p. 23] 

2 . Number 4 . "End the practice of awarding business on the 
basis of price tag. Instead, minimize total cost. Move 
toward a single supplier for any one item, on a long- 
term relationship of loyalty and trust." [Ref. 20:p. 
23 ] 

The first concept addresses the requirement for contractors to 
practice statistical process control (SPC) . The second 
addresses the relationship between buyers and suppliers. 
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Implementation of it involves considering total cost when 
awarding contracts by ensuring that suppliers practice TQM. 

RYG does not satisfy all of the requirements of these 
two principles, nor does it require contractors to practice 
all of Deming 's 14 principles. It does, however, help the 
Government end the practice of awarding business on the basis 
of price tag by considering the total cost of a procurement 
when the cost of poor quality is added to the source selection 
formula. Therefore, in a limited way, RYG helps to further 
the goals of TQM. 

This researcher believes that a limited implementation 
of TQM in field contracting is probably a more realistic 
undertaking for DOD. Even the third and fourth Deming 
principles will be very difficult to implement since they 
require two resources that may not be available to field 
contracting of f icers--time and personnel. 

Time is required to ensure that the supplier practices 
TQM. Contracting personnel would have to examine the TQM 
programs of offerors in detail. They would need to ensure 
that the contractors practice SPC as well as other Deming 
principles. Implementation of the management principles is 
more difficult and more time consuming because it involves 
evaluating the firm's daily management style. Field 
contracting requires the processing of many procurement 
actions in a relatively short period of time. Contracting 
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offices do not devote the time necessary to perform such an 
on-going evaluation. 

In addition to time, personnel are required to perform 
a vendor TQM assessment. As discussed in Chapter III, the 
buying activity must be able to evaluate whether the supplier 
is taking the correct measurements at the critical points in 
production and if the sampling technique is proper. This 
requires an in depth knowledge of both the production process 
and statistics. Someone with technical knowledge of the item 
being purchased, perhaps an engineer, would have to be a 
permanent member of the evaluation team. Additionally, a 
statistician would probably be needed. It is unlikely that a 
contracting office would have either the number of personnel 
or skill levels required for the task. 

2 . TOM and RYG — POD 

With the exception of its short-term goals, the 
segments of the DOD TQM Master Plan relating to procurement 
seem to focus on procurement of major systems. This is 
probably due to the dollar values involved in major system 
procurements as well as the degree of management attention 
focused on each program. The expected returns from using TQM 
for major system acquisitions would be greater than for 
smaller procurements. In addition, the time schedule of a 
major system acquisition would more easily allow the 
contractor evaluation required by TQM. Procurements made in 
the field do not involve such high dollar values. Neither do 
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they allow the degree of management attention required by 
procurement of a major system. However, in spite of the fact 
that the Master Plan does not specifically address contractor 
quality programs intended for field contracting, this 
researcher believes that RYG can still be shown to conform to 
the intent of the Master Plan. 

In the short term, the Master Plan discusses enlisting 
defense industry commitment to TQM by using programs already 
in existence within DOD. Additionally, it hopes to employ 
recognition and reward systems based on TQM goals and 
behaviors. [Ref. 26 :p. 10] RYG is consistent with both these 
obj ectives . 

First, RYG employs an existing system, the Product 
Deficiency Reporting and Evaluation Program ( PDREP) , to 
improve the quality of material received by the Navy. It 
expands the application of this system to allow greater 
consideration of contractor past performance in the source 
selection process. Second, RYG could be used to reward 
contractors who provide good quality material to the 
Government. It accomplishes this by considering the cost of 
poor quality when making source selection decisions. Since 
proposals from contractors with good quality history are not 
assessed a Technical Evaluation Adjustment, these contractors 
have a competitive advantage over those with a poor quality 
history . 
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The Master Plan's procurement goal for the mid term 
addresses only major system procurements. It involves getting 
a commitment from DOD's top 25 contractors to implement TQM. 
These contractors receive more than 50 percent of DOD's 
procurement dollars and most of its major system 
contracts. [Ref. 26:p. 6] This goal is not related to RYG 
since RYG neither helps nor hinders its implementation. 

In the long term, the Master Plan seeks employment of 
TQM by all DOD prime and subcontractors [Ref. 26:p. 4]. As 
discussed earlier in this section, it is difficult for field 
contracting offices to require and verify employment of TQM by 
the vast number of contractors dealt with at this level. 
However, RYG at least allows them to consider improved 
quality, which is the result of continuous process 
improvement, in their source selection decisions. Therefore, 
use of RYG is at least a step toward TQM implementation in 
field contracting. Additionally, implementing a Blue Ribbon 
Contractor program in conjunction with RYG can move field 
contracting activities even closer to the goals of TQM. 

As described in the Blue Ribbon Contractor Program 
section of this chapter, the RYG and Blue Ribbon Contractor 
programs can enhance each other. RYG is necessary for 
procurement of infrequently used and non-critical materials 
that do not justify a great deal of management attention. It 
is also needed by commands incapable of implementing a Blue 
Ribbon Contractor program. RYG focuses on the negative 
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aspects of poor performance to improve quality. A Blue Ribbon 
Contractor program requires more resources to implement than 
does RYG and so would be used in the procurement of materials 
an activity considers critical. Blue Ribbon programs focus on 
the positive aspects of past performance to improve quality. 

Because of its simplicity and focus on the negative, 
RYG could be considered a first step toward implementing TQM 
in field contracting. The program will always be needed when 
a Blue Ribbon program is either unavailable or inappropriate. 
Since the Blue Ribbon Contractor programs are more complex and 
focus on the positive, they are a logical second step toward 
implementing TQM in field contracting. 

E . SUMMARY 

This chapter evaluated the three DOD quality programs 
described in the previous chapter--CPARS , the Blue Ribbon 
Contractor program, and the TQM initiative. It compared and 
contrasted these programs to RYG and discussed whether RYG 
should co-exist with them. The next chapter will assess the 
strengths and weaknesses of the RYG program. The analysis 
will include a discussion concerning the program's ability to 
withstand a protest. It will also assess the current status 
of the RYG test. 
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V. STRENGTHS AND WEAKNESSES OF RED YELLOW GREEN 



A. INTRODUCTION 

This chapter will attempt to assess the strengths and 
weaknesses of the Red Yellow Green (RYG) program. First, the 
four areas of the program most open to contractor protests 
will be addressed. The first area considered will be whether 
RYG results in de facto debarment of Red and Yellow contrac- 
tors. The second consideration is whether the Technical 
Evaluation Adjustments (TEAs) could be regarded as penalties 
and whether it is fair for the Government to consider costs of 
quality (a basic premise of RYG) in the source selection 
decision in the first place. The third to be discussed will 
be whether consideration of past performance, as an evaluation 
factor, constitutes a responsibility determination. The 
fourth area addressed in this chapter will be the accuracy of 
the data base used to rate contractors Red, Yellow, or Green. 
The overall discussion will include an evaluation of the 
program's ability to withstand a contractor protest. 

Next, the progress of the RYG program test will be 
examined. This discussion will first involve an assessment of 
test implementation at each of the five test activities. It 
will then provide data documenting the results of the test to 
date and discuss the significance of these data. 
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B. PROTEST ISSUES 



1 . De Facto Debarment 

The discussion concerning de facto debarment will 
center around whether designating a contractor as Red or 
Yellow has the same effect as debarment. The effect of 
debarment as described in the Federal Acquisition Regulation 
(FAR) is: 

(a) Debarred or suspended contractors are excluded from 
receiving contracts, and agencies shall not solicit offers 
from, award contracts to, or consent to subcontracts with 
these contractors, unless the acquiring agency's head or a 
designee determines that there is a compelling reason for 
such action.... [Ref. 28] 

Cibinic and Nash provide a similar description of the effect 
of debarment: 

The second broad sanction is debarment of the contractor 
or its employees from taking any Government contracts for a 
stated period of time .... Debarment precludes contracts with 
any agency of the federal government for a period of up to 
three years.... [Ref. 29] 

Based on the above definition, it is clear that debarment of 
contractors excludes them from competing for any Government 
contract. This means that the Government must avoid both 
soliciting offers from and awarding contracts to debarred 
contractors . 

Application of the RYG test procedures neither stops 
the Government from soliciting offers from nor awarding 
contracts to Red and Yellow contractors [Ref. 30]. It merely 
allows the Government to consider the costs it expects to 
incur based on the quality of material it is most likely to 
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receive from an offeror. If, after considering the costs 
related to additional quality assurance requirements, a Red or 
Yellow contractor is the lowest-priced responsible offeror, 
the Red or Yellow contractor will be awarded the contract. 

Since the Government will be soliciting proposals 
from, and awarding contracts to, Red and Yellow contractors, 
application of the RYG test procedures is not considered a de 
facto debarment in the analysis of the researcher. It is 
expected that the RYG program will be able to withstand any 
protest in this area. 

2 . Use of TEAS 

The second area for potential protest is the use of 
TEAs. The propriety of using TEAs will be considered first 
by discussing whether TEAs could be considered penalties and, 
therefore, unenforceable, and second by discussing whether it 
is fair for the Government to consider costs of quality when 
evaluating proposals. In order to assess the reasonableness 
of the values assigned as TEAs, the researcher considered 
precedent associated with the reasonableness of liquidated 
damages amounts. 

Regulations and precedent applying to liquidated 
damages were used to determine whether the TEAs (as calculated 
in the RYG test procedures) could be considered penalties. 
The FAR states that "liquidated damages fixed without any 
reference to probable actual damages may be held to be a 
penalty, and therefore unenforceable." [Ref. 28:para. 12.202] 
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Based on this regulation, the question then becomes whether 
the TEAs are related to actual costs that the Government would 
expect to incur when forced to carry out additional quality 
assurance requirements. 

Cibinic and Nash state, "The Government contract rule 
is that the reasonableness of the forecast of liquidated 
damages is evaluated by looking at the situation at the time 
the contract is made...." [Ref. 29 :p. 802] They further 
explain that even liquidated damages calculated at a fixed 
rate of dollars per day have been upheld by the courts because 
they are considered to cover the Government's administrative 
expenses, "i.e., inspection, superintendence or engineering 
costs, which the Government will incur if a delay is 
encountered." [Ref. 29:p. 803] In the case of Young 
Associates, Inc. v. United States, 200 Ct. Cl. 438,471 F.2d 
618 (1973) the court stated: 

The answer is, we think, that the regulation does not 
require a 1 iquidated-damage schedule to be tailor made for 
each individual contract. It is enough if the amount 
stipulated is reasonable for the particular agreement at the 
time it is made. [Ref. 29:p. 804] 

In summary, amounts of liquidated damages are 
considered reasonable if they are a fair estimate (calculated 
when the contract is made) of the administrative costs the 
Government would expect to incur during a delay. Further, the 
estimate need not be tailored to each contract. Based on 
this, amounts of TEAs should be considered reasonable if they 
are truly a fair estimate of the administrative costs the 
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Government expects to incur because a contractor has poor 
quality history. 

The TEAs furnished as examples in Chapter II and 
Appendix D were calculated using Navy Material Quality 
Assessment Office (NMQAO) historical data. These data include 
labor hours required for the quality assurance actions, labor 
rates, and costs of testing. [Ref. 2] They are, however, 
provided only as examples. Each test activity has used these 
examples and their own quality assurance history and labor 
rates to calculate local TEAS. Therefore, the TEAs assessed 
in the test of RYG are tailored to the expected costs of each 
activity. Since the TEAs represent a fair estimate of the 
costs each activity expects to incur because of increased 
quality assurance requirements (calculated at the time of 
contract award) , this researcher believes that they would be 
considered reasonable in the event of a protest. In addition, 
since the TEAs assessed correspond to the specific quality 
assurance actions required in the contract, they could be con- 
sidered to be tailored to each contract. Based on the case 
law concerning liquidated damages, a tailored assessment might 
not be necessary, however, it probably strengthens the 
argument justifying the reasonableness of assigned TEAs. 

The second consideration in the assessment of the 
propriety of using TEAs is whether it is fair for the 
Government to consider the costs of additional quality 
assurance requirements in source selection. There is 
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precedent for considering costs similar to TEAs in the source 
selection decision. Specifically, the FAR discusses the 
consideration of first article test and transportation costs 
in the source selection process. 

Regarding first article testing, the FAR states, 
"Solicitations containing a testing and approval requirement 
shall ... include, when the Government is responsible for first 
article testing, the Government's estimated testing costs as 
a factor for use in evaluating offers...." [Ref. 28:para. 
9.306] In addition, the FAR requires that the solicitations 
"inform offerors that the requirement may be waived when 
supplies identical or similar to those called for have 
previously been delivered by the offeror and accepted by the 
Government...." [Ref. 28: para. 9.306] These two statements 
suggest that not only should the Government consider the costs 
of testing in the source selection decision, but that the 
decision concerning the requirement of first article testing 
should be based on the performance history of the offerors. 
Both support the use of TEAs when evaluating past performance. 

Regarding transportation costs, the FAR states, 
"Solicitations, when appropriate, shall specify that offers 
may be f.o.b. origin, f.o.b. destination, or both; and that 
they will be evaluated on the basis of the lowest overall cost 
to the Government." [Ref. 28: para. 47.305-2] The FAR further 
states, "When evaluating offers, contracting officers shall 
consider transportation and transportation-related 
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costs..." [Ref. 28:para. 47.306] These statements strengthen 
the argument that it is fair for the Government to use TEAs 
based on differences in the costs it will incur because of 
quality differences. 

3 . Use of Responsibility-Related Factors 

The argument against use of past performance as a 
factor in source evaluation states that since past performance 
is one of seven responsibility factors listed in the FAR, its 
use in source evaluation constitutes a responsibility 
determination [Ref. 28:para. 9.104-1]. In the case of small 
businesses, a determination of non-responsibility must be 
referred to the cognizant Small Business Administration (SBA) 
Regional Office [Ref. 28:para. 19.602-1]. Therefore, if use 
of past performance in source evaluation were considered to 
constitute responsibility determination, any negative impact 
the RYG test procedures had on small business would have to be 
referred to the SBA. Failure to comply with FAR Part 19.602- 
1 would violate statute and regulation. In order to determine 
whether the responsibility argument is valid, several 
pertinent General Accounting Office (GAO) protest decisions 
have been examined. The cases and their implications are 
summarized below. 

Three of the cases evaluated indicate that 
responsibility factors may be used in source evaluation 
provided that negotiated procurement practices are used, and 



65 



the selecting activity uses these factors to make a 
comparative evaluation of offerors. 

In the matter of B & W Service Industries, Inc. (B & 
W) , B & W contended that responsibility factors (i.e., company 
experience, past performance, and the experience of key 
personnel) should not have been used in the Department of 
Housing and Urban Development's evaluation of their proposal. 
GAO denied the protest. [Ref. 31] In its decision, GAO 
stated: 

Contracting officers evaluate prospective contractors to 
determine their responsibility, that is, their capability to 
perform the work....COC (Certificate of Competency) 
referrals to SBA are only required where contracting 
officers find small businesses to be nonresponsible . In 
this case, the agency did not find the protester 
nonresponsible, but considered its proposal to be weak under 
some evaluation factors listed in the solicitation. 

With regard to these factors, it is not improper in a 
negotiated procurement to include traditional responsibility 
factors among the technical evaluation criteria .... As long 
as the factors are limited to areas which, when evaluated 
comparatively, can provide an appropriate basis for a 
selection that will be in the government's best interest, 
COC procedures do not apply to a technical proposal 
deficient in those areas. [Ref. 31] 

In the case of Utah Geophysical Inc. (UGI) , UGI 
contended that the Nuclear Regulatory Commission (NRC) failed 
to refer a non-responsibility determination to the SBA. They 
argued that because source evaluation included responsibility 
factors (i.e., experience and level of effort for key 
personnel assigned to the project), the NRC ' s finding that 
their proposal was unacceptable constituted a determination of 
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non-responsibility. [Ref. 32] GAO denied the protest and 
stated in its decision: 

Matters which are normally considered in responsibility 
determinations may properly be considered in evaluation of 
proposals when negotiation procedures are used and agency 
makes relative assessment of competing offerors' ability to 
meet contract requirements. [Ref. 32] 

In the matter of Electrospace Systems, Inc. (ESI), ESI 
protested the Army's use of offerors' experience in source 
evaluation. This evaluation criterion impacted ESI's 
exclusion from the competitive range. They argued that use of 
responsibility-related factors in source evaluation violated 
the SBA's statutory authority to certify the responsibility of 
small businesses. [Ref. 33] In its decision, GAO cites a 
previous case involving SBD Computer Services Corporation, B- 
186950, December 21, 1976, 76-2 CPD 511. In its 1976 
decision, GAO stated: 

In many other cases, we have recognized that contracting 
agencies may properly utilize evaluation factors which 
include experience and other areas that would otherwise be 
encompassed by offeror responsibility determinations when 
the needs of those agencies warrant a comparative evaluation 
of those areas. [Ref. 33] 

In its decision to deny ESI's protest, GAO stated: 

Since neither 10 U.S.C. 2304(g) nor applicable regulation in 
any way restrict the "other factors" that may be used by 
agencies in selecting the proposal having the greatest value 
to the Government, we have not prohibited procuring agencies 
from using responsibility-related factors in making relative 
assessments of the merits of competing proposals. [Ref. 33] 

The cases cited above all support use of 
responsibility factors in source evaluation. However, care 
must be exercised to ensure that, if such factors are used, 
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the Government makes a comparative evaluation of offerors, not 
a pass/fail determination. If a responsibility factor is used 
to ascertain the technical acceptability of a proposal, its 
use will be considered to constitute a responsibility 
determination. Two cases providing examples of this 

distinction are described below. 

In the case of Sanford and Sons Company (Sanford) , 

Sanford protested the Army Corps of Engineers' determination 

that their proposal was technically unacceptable. The request 

for proposals (RFP) stated that the contract would be awarded 

to the lowest-priced, technically-qualified offeror. Two of 

the six criteria for source evaluation detailed in the RFP 

were: (1) organization and administration; and (2) 

satisfactory performance record. The Army found Sanford's 

proposal to be technically unacceptable in both organization 

and administration and past performance. Both of these areas 

are considered responsibility factors. GAO sustained the 

protest. [Ref. 34] In its decision GAO stated: 

We have cautioned that an agency may not find that a small 
business is nonresponsible under the guise of a relative 
assessment of responsibility factors and thus avoid 
referring the matter to the Small Business Administration. 
[Ref. 34] 

The decision further states: 

Although an agency may use traditional responsibility 
factors, like prior performance, as technical evaluation 
factors where its needs warrant a comparative evaluation of 
proposals, an agency's rejection of a small business firm's 
offer as unacceptable under such factors was improper where 
the agency's decision did not reflect a relative assessment 
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of the offer but instead effectively constituted a finding 
of nonresponsibility. [Ref. 34] 

In the matter of Siems International Electron 

Microscope Service (Siems) , Siems protested the cancellation 

of a RFP by the National Institutes of Health (NIH) . NIH had 

determined that Siems was non-responsible under the RFP, but 

Siems was subsequently issued a Certificate of Competency by 

the SBA . NIH stated that the solicitation was cancelled 

because the specification in the RFP was inadequate, not 

whether or not Siems was responsible. NIH determined, 

therefore, that the requirement should be resolicited. Under 

the new RFP, Siems proposal was determined to be technically 

unacceptable. This determination was based on a 

responsibility factor— contractor capacity. [Ref. 35] GAO 

sustained the protest, stating: 

Where offerors were not required to submit technical 
proposals to service electron microscopes but only to offer 
to conform to the best practices of the industry, and the 
factors making up the technical criteria were evaluation of 
capacity factors, the determination an offeror was 
technically unacceptable amounted, in essence, to a 
determination of nonresponsibility for reasons of capacity 
that required a referral to the Small Business 

Administration.... [Ref. 35] 

The conclusion that can be drawn after examination of 
the above five cases is that responsibility factors may be 
used in source evaluation, but that activities must be careful 
not to use such factors to determine the technical 
acceptability of a proposal. If implemented as written, the 
RYG test procedures do not use past performance to determine 
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technical acceptability. It will be used, however, to better 
determine the true cost to the Government of doing business 
with each offeror. In fact, the test procedures specifically 
state that a contractor's status under the test is not to be 
used as a substitute for the requirement to make a 
responsibility determination [Ref. l:encl. (1), p. 4]. The 
test activities, however, must exercise care in administering 
the RYG procedures to ensure that they do not use a 
contractor's past performance to judge his technical 
acceptability . 

In summary, the Government faces a risk that GAO may 
sustain a protest arguing that use of the RYG test procedures 
constitutes a responsibility determination. However, if the 
test procedures are implemented as intended, the Office of 
General Council (OGC) in the Office of the Assistant Secretary 
of the Navy (Shipbuilding and Logistics) (ASN(S&L)) believes 
that the Government has a good argument to support its case 
that use of past performance under the RYG test constitutes a 
comparative evaluation of offerors, not a responsibility 
determination [Ref. 30]. 

4 . Accuracy of the Data Base 

The validity of the Contractor Evaluation System (CES) 
data base presents the greatest potential for protest since 
they are the basis of categorizing contractors as Red, Yellow, 
or Green. The data are maintained very well by the Naval 
Material Quality Assessment Office (NMQAO) . Therefore, the 
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CES is an accurate portrayal of the data received by NMQAO 
from the sources described in Chapter II. [Ref. 36] Problems 
will occur, however, if the data are inaccurate when they are 
submitted to NMQAO. Culpability (Government or contractor) 
for rejects and deficiencies is assigned before the data are 
submitted to NMQAO. Because of the volume of data received, 
it would be impossible for NMQAO to verify each item [Ref. 
36]. If an activity submits inaccurate data, a contractor 
could be mistakenly classified Red or Yellow. If a contractor 
were to lose a contract because of an incorrect 
classification, he would have a strong argument to support 
filing a protest. 

Two issues arise in evaluating the probability that 
the RYG test procedures will survive a protest based on 
inaccurate data. First, the current accuracy of the data base 
must be considered. Prior to NMQAO classifying contractors 
Red, Yellow, or Green, letters were mailed to all Red and 
Yellow contractors informing them of their status and the 
reasons for the classification. With each monthly update of 
the data base, contractors are notified of any change in their 
status under the test. Contractors were advised to contact 
NMQAO if they had any questions about the program or their 
color classification. NMQAO mailed 2037 notification letters 
and has received 247 oral and written responses to the 
classifications assigned. Thirteen of these responses 
resulted in changes to the data base. In three cases, data 
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corrections resulted in a change of color classification. 
[Ref. 37] Thirteen corrections out of 2037 would suggest a 
0.56 percent error rate. However, such an assumption would be 
invalid since only a small percentage of the contractors 
notified provided a response. 

The only way to estimate the accuracy of the data base 
is to draw a representative sample of contractors and verify 
the data leading to their color classification. Such an 
undertaking would require more resources than are currently 
available to NMQAO. Additionally, even if such an evaluation 
were to be made, there is no yardstick available to determine 
what would be an acceptable level of accuracy. [Ref. 36] 
Therefore, the validity and acceptable accuracy level of the 
data base can only be determined during the test. 

One of the benefits of gaining approval from the 
Defense Acquisition Regulatory Council for the one-year test 
of the RYG program is that GAO has traditionally declined to 
hear protests on other test programs [Ref. 38]. It is hoped 
that a test period free of protests will allow NMQAO to assess 
the validity of the data and resolve inaccurate data, and 
hopefully expand the program to all of DOD. 

The second issue pertaining to the accuracy of the 
data base concerns the notification letters sent to 
contractors informing them of their status in the program. 
These letters inform contractors of their current color 
classification based on documented performance history. In 
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the opinion of Ms. Sandra Desbrow-Jensen of the OGC at 
ASN(S&L), if a contractor does not respond within a reasonable 
period to correct errors in the data base, he will be estopped 
from later protesting the validity of the data base [Ref. 30]. 
If GAO supports this position, the contractor notification 
letters should help eliminate protests based on the accuracy 
of the data. 

C. PROGRESS OF RYG TEST 
1 . Implementation 

In order to assess the progress of the RYG program 
implementation, the researcher conducted interviews with 
personnel at each of the five test activities. The purpose of 
the interviews was to gain the perspective of both contracting 
and quality assurance personnel involved in RYG 
implementation. Therefore, interviews were conducted with 
individuals from both areas at each activity. 

The following paragraphs summarize the comments 
received during the interviews. The perspective of 
contracting personnel are presented first, followed by 
comments from the quality assurance community. Finally, the 
researcher discusses similarities and differences between 
these comments and their implications. 

a. Contracting 

(1) Naval Supply Center (NSC) Pensacola . The 
researcher interviewed CDR Walsh, Contracting Officer at NSC 
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Pensacola when the RYG implementation began. He explained 
that the NSC had begun RYG implementation with a small number 
of Federal Supply Classes (FSCs) . The Contracting Department 
chose 20 FSCs from a list given to them by the Naval Aviation 
Depot (NADEP) Pensacola. Initially, they have applied the RYG 
test procedures to ten of the 20 FSCs and small purchases. 
They intend to later expand the program to all 20 FSCs and 
major procurements. The test has not generated a great deal 
of activity to date, but CDR Walsh anticipates a greater 
impact when the program is expanded. [Ref. 39] 

The test procedures have not yet resulted in 
significant extra work. Quality assurance personnel at the 
NADEP have helped absorb some of the extra burden. The NSC 
has not received much negative feedback from contractors. 
[Ref. 39] 

CDR Walsh expressed concern about the 
accuracy of the data base used for the RYG test. 
Additionally, he worries about the affect the program will 
have on distributors who are also small businesses. Although 
theoretically distributors should be responsible for the 
quality of material they sell, small businesses may not have 
the capability to properly test their stock. He fears that 
the RYG program may hurt these small businesses. [Ref. 39] 

(2) NSC Charleston . Mrs. Davidson, Deputy 
Director, Purchase Division, Regional Contracting Department, 
NSC Charleston, was interviewed to gain the perspective of 
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contracting personnel at the other NSC involved in the RYG 
test. She explained that the RYG test procedures are being 
applied to procurements of Controlled Industrial Material 
(CIM) for the Charleston Naval Shipyard. All solicitations 
for CIM are done in writing, including small purchases, so it 
takes 30 days to receive proposals. As a result, no awards 
have yet been made using the RYG test procedures. [Ref. 40] 

Mrs. Davidson believes that evaluation of 
proposals using the test procedures will involve extra work 
for contracting personnel. However, since no evaluations have 
yet been performed, she does not know how burdensome this 
extra requirement will be. She thinks the program will be 
beneficial because it will allow the NSC to resolve quality 
problems before contract award; therefore, she expects RYG to 
reduce contract administration activities. [Ref. 40] 

Mrs. Davidson expressed concern that 
application of the RYG procedures would have a negative impact 
on the department's Productive Unit Resourcing System (PURS) 
statistics because it adds work before contract award which 
will not be given appropriate credit. Time savings are not 
realized until the contract administration phase of the 
procurement process. Therefore, although she anticipates 
benefits from the program, she wonders whether they will 
outweigh the program's negative impacts. [Ref. 40] 

(3) Ships Parts Control Center CSPCC) . The 
researcher interviewed Mr. Minahan, Director of the Hull, 
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Mechanical, and Electrical Contracting Department, SPCC. He 
explained that SPCC has applied the RYG test procedures to 
five FSCs. So far, his department has not been burdened with 
much extra work. [Ref. 41] 

Mr. Minahan indicated that the procurements 
included in the RYG test at SPCC involve fairly high dollar 
values. As a result, he does not believe the application of 
TEAs will have much impact on their source selection 
decisions. He thinks the TEAs may have a greater effect at 
other activities making smaller dollar value procurements. 
[Ref. 41] 

( 4 ) Naval Air and Engineering Center fNAEC] . Mr . 
Fackenthal, Supervisory Contract Specialist at NAEC Lakehurst, 
was interviewed. He explained that NAEC is applying the RYG 
small purchase method as outlined in the test procedures. 
However, they requested and received approval to follow the 
Fixed Price-Greatest Value procedures instead of the 
procedures using TEAs for their major procurements. NAEC made 
this request after reviewing a sample of the contractors they 
have done business with during the last two years. Their 
review indicated that only ten percent of the contractors in 
the sample were in the CES data base. NAEC attributes this to 
the nature of their procurements. They make very few 
repetitive buys. Many of their procurements have production 
lead times of up to one year. Therefore, the performance 
history generated is too old to classify a contractor as Red, 
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Yellow, or Green. NAEC believes that the more subjective 
Fixed Price-Greatest Value procedures will better suit their 
procurements. [Ref. 42] 

Mr. Fackenthal indicated that it is difficult 
to assess the results of the program yet because they only 
began using the small purchase procedures in October 1989 and 
have not yet made a contract award using the major purchase 
procedures. However, he stated that all the comments he has 
received about the program, including those from contractors, 
have been positive. He indicated that the contracting 
personnel at NAEC are positive about the RYG program. They 
like the idea of being able to use past performance in source 
evaluation. He foresees that the program will encourage 
contractors with poor performance records to improve the 
quality of the material they deliver to the Government. [Ref. 
42] 

(5) Naval Avionics Center (NAC) . The researcher 
interviewed Mr. Wilson, Manager of Acquisition and Improvement 
at NAC. He commented that the RYG test was progressing very 
well at NAC, and that they are considering expanding the 
number of FSCs procured under the program. They are trying to 
tailor the test procedures to their operation. [Ref. 43] 

Mr. Wilson indicated that the RYG test has 
not resulted in a great deal of extra work for contracting 
personnel at NAC. He believes that the ease with which they 
implemented the program is due in part to the culture at NAC 
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which welcomes innovation. 



He would like to see the CES 



expanded to the Buyer Information System described in Chapter 
II. He hopes that the validity of the data base can be proven 
during the test so that in the future it can be used by 
contracting personnel to perform responsibility 
determinations. [Ref. 43] 

Mr. Wilson explained that most of NAC ' s 
contractor quality problems have been eliminated by 
implementation of the Blue Ribbon Contractor program. 
Therefore, they have not been able to determine whether the 
RYG test program has caused contractors to improve the quality 
of the material they supply to NAC. He also indicated that 
field activities can make errors impacting on a contractor's 
color classification. In one instance at NAC, some test 
equipment was improperly programmed. The error resulted in 
the erroneous rejection of material supplied by a vendor on 
NAC ' s Blue Ribbon Contractor list. The rejects at NAC caused 
the contractor to be classified Red in the RYG data base. The 
contractor did not lose any contracts because of his incorrect 
color classification. However, it took NAC a couple of months 
to investigate and correct the problem after the contractor 
questioned his color classification. [Ref. 43] 
b. Quality Assurance 

(1) NADEP Pensacola . Mr. Hargett, Quality 
Developmental Project manager at NADEP Pensacola, was 
interviewed to gain perspective on the progress of the RYG 
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test implementation in Pensacola. Mr. Hargett explained that 
Pensacola had purposely limited the initial application of the 
RYG test procedures to allow personnel involved to become 
accustomed to them. Their slow start has resulted in little 
activity from the test to date. However, involvement in the 
test program has caused them to increase the amount of guality 
data submitted to NMQAO. As a result, they have added 
approximately 1000 vendors to the CES data base. He expects 
this increase to cause more activity under the test in the 
future. [Ref. 44] 

Mr. Hargett indicated that the RYG test 
procedures increase the workload of the quality assurance 
personnel, but they do not mind devoting the required time. 
Because of the test, Mr. Hargett has become deeply involved 
with the contracting personnel at NSC Pensacola. He said this 
is the first time the two organizations have worked together. 
Mr. Hargett has been involved with the RYG program since its 
conception. Although he was positive about the test at the 
outset, he stated that he is even more positive about the 
program now. [Ref. 44] 

(2) Charleston Naval Shipyard . The researcher 
interviewed Ms. Cassell, Supervisory Quality Assurance 
Engineer at the Charleston Naval Shipyard. She indicated that 
the test is beginning slowly there. She believes that most of 
the delay has been caused by unrealistically low cost 
estimates placed on purchase requests by the Shipyard's 
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material planners. She explained that often the planners 
estimate costs to be less than $2500. As a result, the 
purchase requests are not included in the RYG test. When 
material is procured, however, the actual costs are usually 
greater than $2500. Therefore, procurements that should 
qualify for the RYG test are not included because of 
inaccurate cost estimates. She indicated that the Quality 
Assurance Department at the Shipyard has been working with the 
Planning Department to correct this problem. [Ref. 45] 

Ms. Cassell is positive about the RYG 
program. She believes it highlights the Shipyard's quality 
assurance problems to the contracting personnel at NSC 
Charleston. She also thinks, however, that the location of 
contracting and quality assurance in different commands causes 
the Contracting Department at the NSC to be out of touch with 
the quality problems experienced by the Shipyard. She 
believes this separation may hinder the implementation of the 
RYG test. [Ref. 45] 

(3) SPCC . The researcher interviewed Mr. Jornov, 
Quality Assurance Specialist, Level 1/SS Section, SPCC. He 
indicated that SPCC has used the RYG procedures in 
approximately 30 solicitations, but has not yet received any 
material under the program. So far, they have not received 
any complaints from contractors. He explained that they have 
limited the implementation to cover only spare parts. They 
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later expect to expand the program to include valves, their 
major commodity. [Ref. 46] 

Mr. Jornov said that the increased work load 
has not been burdensome for them, involving approximately one 
hour each day. However, the work load will increase if the 
program is expanded. He thinks the RYG program is worthwhile. 
He believes it needs some fine tuning, but that eventually it 
will encourage vendors to improve the quality of their 
material. He indicated that he has misgivings about the data 
base. He is concerned that some activities may not send data 
to NMQAO and that NMQAO may not enter all the data received. 
He fears that an incomplete data base may distort contractor 
color classifications. [Ref. 46] 

(4) NAEC . Mr. Robert Armitage, Quality Assurance 
Specialist, Procurement Division, Quality Assurance 
Department, NAEC, was interviewed. He indicated that the 
extra work required by the RYG test has not been too 
burdensome for quality assurance personnel at NAEC. He is 
very positive about the program. He believes it is a useful, 
viable tool for NAEC. He would like to see the impact the 
program has on contractors with poor quality history 
strengthened. However, he is glad that the program allows 
NAEC to consider quality history to a greater degree than they 
had in the past. He said that any new tool that helps them do 
their job better is welcome. [Ref. 47] 
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(5) NAC . The researcher interviewed Ms. Powell, 
Electronics Engineer, Quality Assurance Department, NAC. She 
indicated that the bulk of the extra work created by the RYG 
test has been related to handling contractor complaints. 
Investigating complaints generally takes between one and two 



weeks . 


Ms. Powell 


believes 


the 


program 


is an excellent 


concept . 


It forces 


commands 


to 


correct 


their data. 


In 


addition , 


before the 


RYG test. 


contractors 


did not know 


(or 



care) whether their quality history was good. Now they do. 
Implementation forces Government activities to communicate 
with contractors. [Ref. 48] 

Ms. Powell indicated that some of the 
contractor complaints have proven to be valid. Therefore, she 
believes the weakest part of the program is the data base. 
Specifically, she thinks problems may occur in placing 
liability (Government or contractor) for quality deficiencies. 
[Ref. 48] 

c. Analysis of Interviews 

None of the people interviewed exhibited a 
negative attitude concerning the RYG test program. In 
general, quality assurance personnel appeared to be more 
positive about the program than contracting personnel. 
Although the test requires extra work of both communities, 
quality assurance personnel seemed more convinced that the 
benefits derived from the program would outweigh the extra 
effort. Perhaps their perspective results from a feeling that 
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for the first time they have a tool to influence the quality 
of material before it is received. The amount of inspection 
and documentation quality assurance performs varies inversely 
with the quality of material received. Therefore, they 
probably perceive that improved quality will simplify receipt 
inspection and reduce documentation of quality deficiencies. 
This will make their job easier. 

Two of the people involved in contracting, 
however, were as positive about the program as were quality 
assurance personnel. These two people work at the two Navy 
Industrial Fund activities involved in the test, NAC 
Indianapolis and NAEC Lakehurst. The two contracting people 
from the NSCs at Pensacola and Charleston were the two most 
concerned about the program. One of the reasons for the 
difference in opinion on this program was based on the PURS 
used at NSCs. As explained by Mrs. Davidson of NSC 
Charleston, this system rewards the NSC for the speed with 
which they award contracts and purchase orders. The RYG test 
procedures slow down the award process, promising improved 
quality at the end of the procurement process. Therefore, 
implementation of the RYG program may penalize activities 
subject to PURS guidelines. A second reason for the 
difference in attitudes may be the location of the contracting 
and quality assurance personnel at each test site. 
Contracting personnel at NSC Pensacola must work with quality 
assurance personnel at NADEP Pensacola; contracting personnel 
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at NSC Charleston must work with quality assurance personnel 
at the Charleston Naval Shipyard. The NSCs and industrial 
activities they support have different missions and different 
commanding officers. These distinctions probably make it more 
difficult for the two functional areas to understand the 
other's concerns and to make the RYG program succeed. 

Aside from helping the activities receive better 
quality material, the interviewees foresee two peripheral 
consequences of the RYG program that are positive. First, 
they expect the program to encourage Red and Yellow 
contractors to improve the quality of material they supply to 
the Government. Second, they indicated that implementation of 
the test procedures has forced contracting and quality 
assurance personnel to work together. The increased contact 
is expected to help the two groups better understand each 
other's problems. The end result should be that both groups 
are able to do their jobs more effectively. 

The biggest area of concern voiced by the test 
activities is the validity of the data base. This worry was 
expressed by NSC Pensacola, both quality assurance and 
contracting at NAC Indianapolis, and quality assurance at 
SPCC. This concern corresponds to the assessment of personnel 
at the Naval Supply Systems Command and OGC at ASN(S&L) 
(described earlier in this chapter) that the greatest 
potential for protest of the program lies with the accuracy of 
the data base. 
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Test Results 



2 . 

The one-year test of the RYG program began 1 August 
1989. Not all of the five activities began using the RYG test 
procedures on this date, however. Moreover, when they began 
the test, several of the activities limited the initial number 
of procurements included in the test. Limited implementation 
allows the activities to fine tune their internal procedures 
before attempting full-scale implementation. Although this 
approach is based on sound judgment, it has the peripheral 
effect of limiting the amount of available data documenting 
the results of the RYG test. 

The researcher has collected the data that are 
currently available; they are summarized below. Although the 
data are scant, an attempt has been made to analyze them and 
assess their implications on the progress of the RYG test. 
The data pertaining to the number of Red (R) , Yellow (Y) , 
Green (G) , and Insufficient Data (I) contractors will be 
presented first. Next, data concerning the status of 
procurements under the program will be reported. 

Table 1 summarizes the numbers of letters sent to 
contractors advising them of their initial color 
classification and subsequent status changes and the responses 
received by NMQAO. The data in Table 1 indicate that the 
initial color classification resulted in 1033 Red and Yellow 
contractors. Subsequent monthly updates to the RYG data base 
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TABLE 1 



RED YELLOW GREEN NOTIFICATION AND RESPONSE 



Month 


Notification Sent 


Response Received 


May 


1033 


179 


July 


653 


23 


August 


348 


35 


September 


273 


10 


Total 


2307 


247 


Source: 


[Ref. 37 : p . 9] 





changed contractor color classifications. Each update has 
resulted in fewer contractors changing status. 

Table 2 describes the categories into which the 
responses fell and the action taken by NMQAO in reaction to 
them. The data in Table 2 indicate that contractors disagreed 
with the data or their color classification in 49 cases (40 
disputed the data and nine disagreed with their color 
classification) . NMQAO determined that their disagreements 
were valid and changed the data in 13 cases. Three of the 13 
data base corrections resulted in a change in color 
classification. Although 2307 contractors were notified, only 
247 (or 10.7 percent) responded. Therefore, the errors 
discovered because of these responses cannot be used to 
estimate the accuracy of the data base. 
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TABLE 2 



RESPONSES — CATEGORIES AND ACTION TAKEN 



Category of Response Quantity 

Information 141 

Unaware of Rejects 52 

Disputed Data 40 

Classification Disagreement 9 

Action Taken Quantity 

No Action 83 

Forwarded Data Profiles 108 

Referred to Other Command 48 

Conducted/Scheduled Meeting 5 

Changed Data 13 

Changed Classification 3 

Source: [Ref. 37:p. 9] 



The first group of procurements made under the RYG 
test are summarized in NMQAO ' s September 1989 status report. 
The data relating to these procurements are detailed in Table 
3. The data in Table 3 indicate that in most cases the 
lowest-priced proposals were submitted by Green and 
Insufficient Data contractors. Of the 77 procurements made 
under the RYG test, Red and Yellow contractors submitted the 
lowest-priced proposals in only ten instances (seven displace- 
ments of Red and Yellow contractors plus three awards to 
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TABLE 3 



RED/YELLOW/GREEN PROCUREMENTS 



Activity 
NAEC Lakehurst 
NAC Indianapolis 
SPCC Mechanicsburg 
NSC Charleston 
NSC Pensacola 
Total 

Source: [Ref. 



Awards to RYGI 
Contractors 

RYGI 

0 0 0 0 

0 3 27 17 

0 0 18 0 

0 0 0 0 

0 0 0 12 

0 3 45 29 

37:pp. 1-6] 



Contractor 

Displacements 

R Y 

0 0 

0 5 

2 0 

0 0 

0 0 

2 5 



Yellow contractors) . These ten cases represent 13 percent of 
the 77 procurements. The awards made to Yellow contractors 
indicate that the Government's consideration of the costs of 
quality does not preclude contractors with poor quality 
histories from receiving Government contracts. In other 
words, a contractor's color classification does not eliminate 
him from the competitive range. 

It is too early to determine whether the RYG program 
will improve either the quality of material the Government 
receives or contractors' motivation to deliver high quality 
material. It is also impossible to judge the reaction 
contractors will have to displacements caused by the program. 
The data do show, however, that the program is having some 
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affect on contract awards; Red and Yellow contractors have 



lost contracts to higher priced contractors with better 
performance histories. The data also show that the test 
activities are not using the RYG data base to eliminate 
contractors from the competitive range. This should support 
the Government's position that application of the RYG formula 
in source evaluations does not constitute a responsibility 
determination . 

D . SUMMARY 

This chapter assessed the strengths and weaknesses of the 
RYG program. To accomplish this, it first discussed the 
ability of the RYG program to withstand protests based on four 
arguments — de facto debarment, TEAs as penalties, 
responsibility determination, and validity of the data base. 
Second, it discussed the progress of RYG test implementation 
at the five test activities. Interviews were conducted with 
personnel involved with the test at each of the activities and 
data related to the test were analyzed. Interviewees 
generally felt that the RYG program would yield positive 
results. However, several of them expressed concern about the 
validity of the data in the CES. The final chapter will 
present the conclusions and recommendations of the thesis. 
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VI . CONCLUSIONS AND RECOMMENDATIONS 



A. INTRODUCTION 

This chapter will discuss the conclusions and recommenda- 
tions resulting from the thesis research. After presenting 
the conclusions and recommendations, the research questions 
will be answered. Finally, recommendations will be made 
concerning areas for further research. 

B. CONCLUSIONS 

1 . The Red Yellow Green (RYG^ Program is Appropriate 
for Use in Field Contracting Only 

The RYG test procedures incorporate a formula driven 
assessment of contractor past performance in source 
evaluation. The formula used in the test was designed to 
assess contractor quality history for groups of relatively 
simple commodities. The test procedures were intended to be 
easy to apply and, therefore, involve very little subjective 
input from contracting and quality-assurance personnel. Major 
system contracting would require a much more subjective 
assessment of contractor-quality history than allowed by the 
RYG program. 

2 . The Air Force Contractor Performance Assessment 
Reporting System (CPARS) is Appropriate for Major 
Systems Procurement Only 

CPARS should not be expanded for use in field 
contracting. Maintenance of the data for CPARS is very labor 
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intensive. The personnel resources required to maintain an 
expanded CPARS would prohibit including enough contractors in 
the system to make it useful for field contracting. In 
addition, use of CPARS in source evaluation requires time and 
personnel not available in field contracting offices. 

3 . Blue Ribbon Contractor Programs are Appropriate for 
Use at Field Activities in Which Contracting and 
Quality Assurance Personnel are Able to Work Together 

Unlike RYG, the assessment of contractor performance 

history under a Blue Ribbon program is not formula-driven. 

The decision to award to a Blue Ribbon contractor over a 

lower-priced offeror (not included in the program) requires 

an assessment by both contracting and quality-assurance 

personnel. Therefore, the two groups must communicate with 

each other to use the program effectively. 

4 . RYG and Blue Ribbon Contractor Programs Can Be 
Implemented Together and Can Enhance Each Other 

A Blue Ribbon program would be used when procuring 

critical goods and services because it allows an activity to 

concentrate its attention on those parameters that it 

considers important. RYG would be used for all other 

procurements which do not justify devotion of the resources 

needed to apply a Blue Ribbon program. 

5 . The RYG Program (Especially if Implemented in Conjunc- 
tion with a Blue Ribbon Contractor Program) is a 
Realistic Method for Initiating Total Quality 
Management (TOM) in Field Contracting 

The time and personnel required to perform vendor TQM- 
assessments make complete TQM implementation in field 
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contracting impractical. However, the RYG and Blue Ribbon 
Contractor programs allow field contracting activities to 
consider contractor past performance in the source selection 
decision. Therefore, they allow activities to comply, at 
least partially, with one of Deming ' s 14 principles: "End the 

practice of awarding business on the basis of price tag. 
Instead, minimize total cost. Move toward a single supplier 
for any one item, on a long-term relationship of loyalty and 
trust." [Ref. 20:p. 23] 

6 . Application of the RYG Test Procedures Does Not 
Constitute De Facto Debarment 

Use of the test procedures neither stops the 
Government from soliciting offers from nor awarding contracts 
to Red and Yellow contractors. If, after considering the 
costs related to additional quality assurance requirements, a 
Red or Yellow contractor is the lowest priced responsible 
offeror, the Red or Yellow contractor will be awarded the 
contract . 

7 . The Assessment of Technical Evaluation Adjustments 
(TEAs ) Under the RYG Test is Reasonable and Does Not 
Represent a Penalty to Red and Yellow Contractors 

There is precedent for considering costs similar to 

TEAs in source evaluation (i.e., consideration of first- 

article test and transportation costs) . Therefore, it is 

reasonable for the Government to consider the costs of 

additional quality assurance requirements in source selection. 

TEAs are calculated using historical data for labor hours 
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required for quality assurance actions, labor rates, and costs 
of testing. Since their calculation is based on actual costs 
the Government would expect to incur when forced to carry out 
additional quality assurance actions, TEAs should not be 
considered penalties. 

8 . If the RYG Test Procedures are Implemented as 
Intended, the Government Should Be Able to Support 
the Argument that the Use of Past Performance Under 
the Test Does Not Constitute a Responsibility 
Determination 

Precedent from cases in which contractors protested 
the use of responsibility-related factors in source evaluation 
indicate that responsibility factors may be used in source 
evaluation. However, activities must be careful to use such 
factors to perform a comparative evaluation of offerors, not 
to determine the technical acceptability of proposals. The 
RYG test procedures use past performance to better determine 
the true cost to the Government of doing business with each 
offeror, not to determine the technical acceptability of 
proposals . 

9 . The Validity of the Contractor Evaluation System (CES) 
Data Base Presents the Greatest Potential for Protest 
of the RYG Test Procedures 

Personnel at the Naval Supply Systems Command and at 
several of the activities implementing the RYG test expressed 
concern regarding the accuracy of the data in the CES . 
Specifically, they fear that if an activity were to submit 
inaccurate data, a contractor could be mistakenly classified 
Red or Yellow. If a contractor were to lose a contract 
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because of an incorrect classification, he would have a strong 
argument to support filing a protest. 

10 . It is Possible that Red and Yellow Contractors Will Be 
Estopped from Protesting the Validity of the CES 

Each Red and Yellow contractor is notified of his 

color classification and the performance history behind the 

designation. He is given an opportunity to challenge the data 

supporting his status under the RYG program. If a contractor 

is aware that the data justifying his classification are 

incorrect, and he does not respond to correct them within a 

reasonable period, he may give up the right to challenge these 

errors later during a protest. 

C. RECOMMENDATIONS 

1 . Evaluation of RYG at the End of the Test Period 

The benefits of the RYG program and its ability to 
withstand a protest should be evaluated at the end of the 
program test, currently scheduled for 1 August 1990. There 
are not enough data currently available to evaluate whether 
the benefits of the RYG program outweigh the costs of 
implementation. The positive comments expressed by personnel 
at the five activities involved in the test suggest that the 
program will be beneficial. However, an assessment concerning 
whether the program improves contractor performance must 
eventually be made. Such an evaluation should include 
reviewing quality records at the five test activities to 
determine whether the quality of incoming material has 
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improved. In addition, a sample of contractors should be 
evaluated to determine whether the RYG program has encouraged 
them to improve their performance. 

2 . Continuation of the RYG Program After Completion of 
the Test 

If there are not enough data to adequately assess the 
value of the RYG program at the end of the test period, use of 
the test procedures should be continued until the program can 
be properly evaluated. Contractor evaluation, as described in 
the previous paragraph, will require the passage of enough 
time for contractors to become aware of the effects of the RYG 
program, change their performance, and deliver enough material 
to the Government to document that change in performance. The 
criteria used to designate contractors as Red, Yellow, and 
Green span periods up to two years. Therefore, it is probable 
that an accurate evaluation of the affect the RYG program has 
on contractors will require more than the one year of data 
that will be accumulated under the test. Since the program 
has the potential to yield such positive results, its use 
should be continued until the program can be evaluated 
properly. 

3 . Implementation of Blue Ribbon Contractor Programs with 
RYG 

If the RYG program is determined to be beneficial, 
activities should be encouraged to implement a tailored Blue 
Ribbon Contractor program in conjunction with RYG. As 
discussed in the Conclusions section of this chapter, RYG and 



95 



Blue Ribbon Contractor programs can enhance each other. In 
addition, they both move field contracting offices toward 
implementation of TQM. Therefore, implementing them together 
should prove beneficial to the Department of Defense (DOD) . 

D. ANSWERS TO THE RESEARCH QUESTIONS 

1 . How is the Navy’s Red Yellow Green Program Structured 
and How is it Intended to Improve the Quality of 
Material Procured by the Navy? 

The RYG program uses contractor performance history in 
source evaluation. The program uses this history in one of 
two ways. The first approach uses a contractor's quality 
history to determine whether the Government will require 
additional quality assurance actions when doing business with 
the contractor. If the contractor's performance history is 
good, additional quality assurance actions will not be 
required; if his history is poor, additional actions will be 
required. If additional quality assurance is required, the 
costs of performance to the Government are added to the 
contractor's proposal using TEAs. The source selection 
decision is made using the proposed prices of Green 
contractors and the adjusted prices of Red and Yellow 
contractors . 

The second approach involves a more subjective use of 
contractor performance history. A source selection plan is 
prepared by quality assurance personnel, specifying the 
criteria to be used in evaluating Red, Yellow, and Green 
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contractors. When proposals are received, quality assurance 
personnel assign ratings to each offeror based on the criteria 
outlined in the source selection plan. Contracting personnel 
award the contract based on the scores received by 
contractors, considering their quality and price ratings. 

The RYG program is intended to improve the quality of 
material procured by the Navy in two ways. First, 
consideration of contractor quality history in source 
evaluation should result in a higher percentage of contract 
awards to contractors with good performance history. This is 
expected to improve the quality of material received. Second, 
the program should encourage contractors with poor quality 
histories to improve their performance in order to receive 
more Government contracts in the future. 

2 . How are Red. Yellow, and Green Contractors Defined? 

A Red contractor is considered a high quality risk. 
His performance history is poor. It warrants requirement of 
special quality assurance actions and higher-level review 
before contract award. A Yellow contractor is considered a 
moderate quality risk. His performance history is poor enough 
to warrant requirement of special quality assurance actions. 
A Green contractor is considered a low quality risk. His 
performance history is good and, therefore, warrants no 
special quality assurance actions. 
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3 . What are the Other Current Contractor Quality 
Initiatives within POD? 

The other current quality initiatives within DOD are 
the Air Force CPARS , the Naval Avionics Center (NAC) 
Indianapolis Blue Ribbon Contractor program, and the DOD TQM 
initiative . 

4 . How Does RYG Compare and Contrast to These Other 
Programs? 

RYG and CPARS were designed to be used in different 
areas of procurement. RYG is intended to be employed by field 
contracting activities. CPARS is intended for major systems' 
procurement. RYG and NAC ' s Blue Ribbon Contractor program are 
both intended for use in field contracting. The Blue Ribbon 
program requires an activity to devote more resources to and 
employ more judgment in contractor performance evaluation than 
does RYG. It would be impractical for some activities to 
implement a Blue Ribbon Contractor program. However, 
implementation of such a program can enhance the results 
expected of the RYG program. The RYG program is consistent 
with the goals of the DOD TQM initiative. It helps field 
contracting offices to begin implementation of TQM. 

5 . What are the Strengths and Weaknesses Associated with 
RYG? 

The RYG program has many strengths. It allows use of 
contractor performance history in the source selection 
decision and so should improve the quality of material 
received by the Government. It requires few resources to 
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implement and, therefore, can be used by field contracting 
offices. It forces contracting and quality-assurance 
personnel to work together. It forces Government activities 
to communicate with industry. The program does have some 
weaknesses, however. The data base used in the program is 
very large, and its accuracy cannot be currently assessed. If 
the data base proves inaccurate, the program is built on a 
weak foundation. The program's evaluation of contractor 
performance history is formula-driven. Therefore, it is only 
appropriate for procurement of relatively simple commodities. 

6 . How is the RYG Program Being Tested in the Market? 

The RYG program is being tested at five Navy field 
activities. Interviews with personnel at the test activities 
indicate that the test began slowly but is progressing well. 
Both contracting and quality assurance personnel expressed 
positive opinions about the program. There are currently too 
few data, however, to assess the success of the test. 

E. AREAS FOR FURTHER RESEARCH 

One area for further research is to evaluate the benefits 
of the RYG program after the one-year test period. More data 
will be available at this point and, therefore, a researcher 
will be better able to assess the merits of the program. 
Another area for further research is to determine how NAC 1 s 
Blue Ribbon Contractor program could be exported to other 
activities. The Blue Ribbon program has proven very 
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beneficial for NAC and could be expected to benefit other 
field contracting activities. 
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APPENDIX A 



RED YELLOW GREEN CLASSIFICATION CRITERIA 



COLOR CODE CLASSIFICATION CRITERIA 

RED A ON CURRENT NAVY VDAR 



B METHOD C, D, AND/OR E CURRENTLY IN EFFECT 

C QUALITY INFO ON LATEST PRE-AWARD SURVEY 

(PAS) WITHIN LAST YEAR — NO AWARD 

D LATEST PRODUCT-ORIENTED SURVEY (POS) IN 

LAST TWO YEARS UNACCEPTABLE 



E LATEST QUALITY SYSTEM REVIEW (QSR) IN LAST 

TWO YEARS UNACCEPTABLE 



F LATEST SPECIAL SURVEY IN LAST TWO YEARS 

UNACCEPTABLE 

G REJECT RATE 15% OR MORE IN LAST TWO YEARS 

FOR FIVE OR MORE LOTS 



H TWO OR MORE UNSAT FIRST ARTICLE TESTS 

(FAT) IN LAST YEAR 

J TWO OR MORE CATEGORY "I" QDRS IN LAST YEAR 

K SIX OR MORE CATEGORY "II" ACTION QDRS IN 

LAST YEAR 



L SIX OR MORE CONTRACTOR LIABLE RODS IN LAST 

year" 

M SIX OR MORE CONTRACTOR LIABLE WAIVERS/ 

DEVIATIONS IN LAST YEAR" 

N ON DLA CONTRACTOR ALERT LIST FOR MAJOR 

DEFICIENCIES" 



Classification criteria L, M, N for "Red" and "Yellow" 
have not yet been incorporated. 
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COLOR CODE 


CLASSIFICATION CRITERIA 


YELLOW A 


ISSUED VDAR LETTER OF CONCERN 


B 


PREVIOUSLY CLASSIFIED "RED" 


C 


LATEST PAS IN LAST TWO YEARS — AWARD WITH 
FINDINGS 


D 


LATEST POS IN LAST TWO YEARS ACCEPTABLE 
WITH CORRECTIONS 


E 


LATEST QSR IN LAST TWO YEARS ACCEPTABLE 
WITH CORRECTIONS 


F 


LATEST SPECIAL SURVEY IN LAST TWO YEARS 
ACCEPTABLE WITH CORRECTIONS 


G 


REJECT RATE 6-14% FOR FIVE OR MORE 
LOTS/ANY REJECT RATE FOR LESS THAN FIVE 
LOTS IN LAST TWO YEARS 


H 


ONE UNSAT FAT IN LAST YEAR 


J 


ONE CATEGORY "I" QDR IN LAST YEAR 


K 


THREE-FIVE CATEGORY "II" ACTION QDRS IN 
LAST YEAR 


L 


THREE-FIVE CONTRACTOR LIABLE RODS IN LAST 

year" 


M 


THREE-FIVE CONTRACTOR LIABLE WAIVERS/ 
DEVIATIONS IN LAST YEAR* 


N 


ON DLA CONTRACTOR ALERT LIST FOR MINOR 

deficiencies" 



"classif ication criteria L, M, N for "Red" and "Yellow" 
have not yet been incorporated. 
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COLOR 


CODE 


CLASSIFICATION CRITERIA 


GREEN 


C 


LATEST PAS IN LAST TWO YEARS — AWARD WITH 
NO FINDINGS 




D 


LATEST POS IN LAST TWO YEARS ACCEPTABLE 




E 


LATEST QDR IN LAST TWO YEARS ACCEPTABLE 




F 


LATEST SPECIAL SURVEY IN LAST TWO YEARS 
ACCEPTABLE 




G 


REJECT RATE LESS THAN 6% IN LAST TWO YEARS 
FOR FIVE OR MORE LOTS 




H 


ALL FAT IN LAST YEAR SATISFACTORY 




K 


ONE-TWO CATEGORY "II" ACTION QDRS IN LAST 
YEAR 




L 


ONE-TWO CONTRACTOR LIABLE RODS IN LAST 
YEAR* 




M 


ONE-TWO CONTRACTOR LIABLE WAIVERS/ 
DEVIATIONS IN LAST YEAR* 



"Classification criteria L and M for "Green" have not yet 
been incorporated. 
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APPENDIX B 



CLAUSES FOR SIMPLIFIED SMALL PURCHASE PROCEDURES 



NOTICE TO PROSPECTIVE OFFERORS (NOV 1988) 

(a) This procurement is subject to a test of the Navy's 
Contractor Evaluation System (CES) , "Red/Yellow/Green" (R/Y/G) 
program. The test is authorized by the Assistant Secretary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 

(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractors' past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. The 
price to be considered in determining best value will be the 
evaluated price after Technical Evaluation Adjustments (TEAs) 
for related quality assurance actions, as applicable, are 
applied to the offered price. 

(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTORS — TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government--price, past quality performance, and 
other factors considered. 

(d) The commodities included in this test, as currently 
solicited, are: 

FSC No. FSC Nomenclature CLIN 
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ADDITIONAL EVALUATION FACTORS — TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (SIMPLIFIED SMALL PURCHASE PROCEDURES) 

(a) This procurement is part of a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics) , for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program's effectiveness and impact on 
the acquisition process. 

(b) The purpose of R/Y/G is to assist contracting personnel 
during source selection to determine the best value for the 
Government — price, past quality performance, and other factors 
considered. The test program uses accumulated contractor 
quality performance data to classify contractor's performance 
on select commodities as either "Red" (high risk) , "Yellow" 
(moderate risk), "Green" (low risk), or "Insufficient Data," 
based on the degree of risk to the Government of receiving 
poor quality products. Such classifications are then used to 
apply a Technical Evaluation Adjustment (TEA) during source 
selection . 

(c) A TEA is a monetary assessment added to the price of 
selected commodities that have been classified as either "Red" 
or "Yellow" for specific contractors, and is based on the cost 
to the Government for effecting additional quality 
considerations that would otherwise not be required if award 
were made to a contractor with a satisfactory performance 
history. For purposes of requirements using the simplified 
small purchasing procedures, standardized TEAS have been 
established for the "Red" and the "Yellow" classifications. 
During evaluation of quotations, the applicable TEA is added 
to the quoted price of the "Red" and/or "Yellow" commodity, 
and after consideration of any other pertinent price-related 
factors (e.g., transportation charges, First Article Testing, 
discount terms, etc.), becomes the basis for determining award 
of the purchase order. A commodity's classification may 
change over time as new or revised quality performance data 
become available. 

(d) Classifications for the test program are summarized as 
follows : 

"Green" — Low risk. No extraordinary quality requirements or 
additional actions required; satisfactory quality history. 

"Yellow" — Moderate risk. History of quality problems; special 
quality requirements/actions needed; Technical Evaluation 
Adjustment (TEA) applied to offered price. 
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"Red" — High risk. Special alert to history of poor quality 
performance; TEA applied to offered price(s), and contract 
award requires higher level approval. 

"Insufficient Data" — Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, a TEA is not assessed. 

(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to; 
Naval Sea System Command Detachment, Naval Material Quality 
Assessment Office (NMQAO) , Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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APPENDIX C 



CLAUSES FOR MAJOR PURCHASE PROCEDURES 



NOTICE TO PROSPECTIVE OFFERORS (NOV 1988) 

(a) This procurement is subject to a test of the Navy's 
Contractor Evaluation System (CES) , "Red/Yellow/Green" (R/Y/G) 
program. The test is authorized by the Assistant Secretary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 

(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractors' past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. The 
price to be considered in determining best value will be the 
evaluated price after Technical Evaluation Adjustments (TEAs) 
for related quality-assurance actions, as applicable, are 
applied to the offered price. 

(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTORS — TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government — price, past quality performance, and 
other factors considered. 

(d) The commodities included in this test, as currently 
solicited, are: 

FSC No. FSC Nomenclature CLIN 



107 



ADDITIONAL EVALUATION FACTORS — TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (MAJOR PURCHASE PROCEDURES) 

(a) This procurement is part of a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics) , for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program's effectiveness and impact on 
the acquisition process. 

(b) The purpose of R/Y/G is to assist contracting personnel 
during source selection to determine the best value for the 
Government — price, past quality performance, and other factors 
considered. The test program uses accumulated contractor 
quality performance data to classify contractor's performance 
on select commodities as either "Red" (high risk) , "Yellow" 
(moderate risk), "Green" (low risk), or "Insufficient Data," 
based on the degree of risk to the Government of receiving 
poor quality products. Such classifications are then used to 
apply a Technical Evaluation Adjustment (TEA) during source 
selection . 

(c) A TEA is a monetary assessment added to the price of 
selected commodities that have been classified as either "Red" 
or "Yellow" for specific contractors, and is based on the cost 
to the Government for effecting additional quality 
considerations that would otherwise not be required if award 
were made to a contractor with a satisfactory performance 
history. During evaluation of quotations, the necessity for 
any additional quality assurance requirements will be 
determined, and the applicable TEA will be assessed onto the 
quoted price of the "Red" and/or "Yellow" commodity. After 
consideration of any other pertinent price-related factors 
(e.g., transportation charges, First Article Testing, discount 
terms, etc.), this adjusted price becomes the basis for 
determining award of the purchase order. A commodity's 
classification may change over time as new or revised quality 
performance data become available. 

(d) Classifications for the test program are summarized as 
follows : 

"Green" — Low risk. No extraordinary quality requirements or 
additional actions required; satisfactory quality history. 

"Yellow" — Moderate risk. History of quality problems; special 
quality requirements/actions needed; Technical Evaluation 
Adjustment (TEA) applied to offered price. 
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"Red"--High risk. Special alert to history of poor quality 
performance; TEA applied to offered price(s), and contract 
award requires higher level approval. 

"Insufficient Data" — Generally, may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, a TEA is not assessed. 

(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Navy Material Quality 
Assessment Office (NMQAO) , Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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APPENDIX D 



CODE 

A 



B 

C 

D 

E 

F 



GUIDELINE FOR TEA ASSIGNMENT 
RED CLASSIFICATION 



ADDITIONAL 

CLASSIFICATION CRITERIA OA REO'TS 



On Current Navy VDAR 


1A 


or 


IB 


(Time Frame — Six Months) 


2 A 


or 

3 


2B 




5 


4 

or 

7 


6 


DLA Corrective Action 


1A 


or 


IB 


Methods C, D, or E Currently 
in Effect 


2A 


or 

4 


2B 


(Time Frame — Within Two Months) 


5 


or 

7 


6 


Latest Pre-Award Survey 
(Negative Quality) 


1A 


or 

4 


IB 


(Time Frame — Two Years) 


5 


or 

7 


6 


Latest Product Oriented Survey 
Rated Unsat — Could Impact 


1A 


or 

4 


IB 


Product Quality 
(Time Frame-Two Years) 


5 


or 

7 


6 


Latest Quality System Review 


1A 


or 


IB 


Rated Unsat — Could Impact 
Product Quality 


2A 


or 

4 


2 B 


(Time Frame — Two Years) 


5 


or 

7 


6 


Latest Quality Audit/Special 
Survey 


1A 


or 

4 


IB 


Rated Unsat--Could Impact 

Product Quality 

(Time Frame — Two Years) 


5 


or 

7 


6 
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CODE 



CLASSIFICATION CRITERIA 



ADDITIONAL 
OA REO'TS 



G 



H 



J 



K 



L 



M 



N 



1A or IB 

3 

4 

5 or 6 
5 or 6 
7 

or IB 
or 2B 
4 

or 6 
7 

I 1A or IB 
2A or 2B 

3 

4 

5 or 6 
7 



Quality Deficiency Reports — 
Cat II 

Six or More "Action" Cat II 


1A 


or 

3 

4 


IB 


(Time Frame — One Year) 


5 


or 

7 


6 


Report of Discrepancy 

Six or More Contractor Liable 


2A 


or 

4 


2B 


RODS 

(Time Frame — One Year) 


5 


or 

7 


6 


Waivers/Deviations 

Six or More Contractor Liable 


2 A 


or 

4 


2B 


Requests 

(Time Frame— One Year) 


5 


or 

7 


6 


DLA Contractor Alert List 


1A 


or 


IB 


Listed for Major Deficiencies 
Which Could Impact Product 
Quality 


2A 


or 

3 

4 


2B 


(Time Frame — Six Months) 


5 


or 

7 


6 



Quality Deficiency Reports — Cat 
Two or More Unsat 
(Time Frame— One Year) 



First Article Test 1A 
Two or More Unsat 2A 
(Time Frame — One Year) 

5 



Material Inspection Records 
Reject Rate 15% or Greater 
Indicated 

(Minimum Five Lots in Two Years) 
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YELLOW CLASSIFICATION 



CODE 

A 



B 

C 

D 

E 

F 

G 

H 



CLASSIFICATION CRITERIA 



ADDITIONAL 
OA REO'TS 



VDAR Discussion Contractor or 
Letter of Concern Issued 
(Time Frame — Six Months) 



1A or IB 

3 

4 

5 or 6 
7 



Previously Classified "Red" 
(Time Frame — Six Months) 



Latest Pre-Award Survey 
Positive for Quality but 
Negative in 

One or More Other Categories 
(Time Frame — Two Years) 

Latest Product Oriented Survey 
Unsat for Minor Problems — No 
Impact on Product Quality 
(Time Frame — Two Years) 

Latest Quality System Review 
Unsat For Minor Problems 
on Product Quality 
(Time Frame — Two Years) 

Latest Quality Audit/Special 
Survey 

Unsat for Minor Problems — No 
Impact on Product Quality 
(Time Frame — Two Years) 

Material Inspection Reports 
Reject Rate Greater Than 5% 
but less than 15% Indicated 
(Based on Minimum of Five Lots 
in Two Years) 

First Article Test 
One Unsat 

(Time Frame — One Year) 



1A or IB 

3 

4 

5 or 6 
7 

1A or IB 
4 

5 or 6 



4 

5 or 6 



4 

5 or 6 



4 

5 or 6 



4 

5 or 6 
7 



2A 

4 

5 or 6 
7 
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ADDITIONAL 


CODE 


CLASSIFICATION CRITERIA 


OA REO'TS 


J 


Quality Deficiency Reports--Cat I 




4 






One Cat I 


5 


or 


6 




(Time Frame — One Year) 




7 




K 


Quality Deficiency Reports — Cat II 




4 






More Than Two but Less Than Six 


5 


or 


6 




Cat II 

(Time Frame — One Year) 




7 




L 


Report of Discrepancy 




2 A 






More than Two but Less Than Six 




4 






Contractor Liable RODs 
(Time Frame — One Year) 




7 




M 


Waivers/Deviations 




2A 






More Than Two but Less Than Six 




4 






Contractor Liable Requests 
(Time Frame — One Year) 




7 




N 


DLA Contractor Alert List 




2A 






Listed for Minor Deficiencies — 




4 






No Impact on Product Quality 
(Time Frame--One Year) 


5 


or 


6 
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ADDITIONAL QUALITY ASSURANCE REQUIREMENTS 



Code and Description TEA 

1 . Pre-Award Survey 

A. DCAS — Quality Survey $ 500 

B. With procurement representative 
participation 

(1) Local 1 775 

(2) Intermediate 2 1,380 

(3) Distant 3 2,095 

2. Post-Award Orientation Conference 

A. DCAS — Quality 550 

B. With procurement representative 
participation 

(1) Local 4 1,075 

(2) Intermediate 5 2,110 

(3) Distant 6 3,590 

3 . Product Oriented Survey — Procurement 
Representative with DCAS participation 

(1) Local 7 o 800 

(2) Intermediate 8 1,500 

(3) Distant 9 2,215 

4 . Government Source Inspection 18 500 

5. Receipt Inspection at Source — Navy 
representative with DCAS participation 

(1) Local 11 650 

(2) Intermediate 12 1,3 60 

(3) Distant" 2,182 

6. Receipt Inspection at Destination — Navy 
representative 

(1) Low 14 597 

(2) Medium 15 1,194 

(3) High 16 2,332 

7. Quality Assurance Letter of Instruction 17 755 

1 ft 

8. Purchase Referrals 640 
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TEA CALCULATIONS 



Calculated $30/hr x 8 hrs = $240 + $35 mileage = $275 + 

500 . 

Calculated $30/hr x 8 hrs = $240 + $240 (8 hrs travel @ 
$30/hr) + $200 (2 days per diem @ $100/day) + $200 travel 
costs = $880 + $500. 

Calculated $30/hr x 8 hrs = $240 + $480 (16 hrs travel @ 
$30/hr) + $300 ( 3 days per diem @ $100/day) = $575 travel 
costs = $1,595 + $500. 

Calculated $30/hr x 16 hrs = $480 + $45 mileage = $525 + 

$550 . 

Calculated $30/hr x 16 hrs = $480 + $480 (16 hrs travel @ 
$30/hr) + $400 (4 days per diem @ $100/day) + $300 travel 

costs = $1,660 + $550. 

Calculated $30/hr x 16 hrs = $480 + $960 (32 hrs travel @ 
$30/hr) + $600 (6 days per diem @ $100/day) + $1,000 travel 
costs = $3,040 + $550. 

Calculated $30/hr x 12 hrs = $360 + $40 mileage = $400 + 
$400 ( DCAS costs) . 

Calculated $30/hr x 12 hrs = $360 + $240 (8 hrs travel @ 
$30/hr) + $300 ( 3 days per diem @ $100/day) + $200 travel 
costs = $1,100 + $400 (DCAS costs). 

Calculated $30/hr x 12 hrs = $360 + $480 (16 hrs travel @ 
$30/hr) + $400 (4 days per diem @ $100/day) + $575 travel 

costs = $1,815 + $400 (DCAS costs). 

10 Calculated $34. 18/hr x 14 hrs. 

“Calculated $43/hr x 8 hrs = $344 + $31 mileage = $375 + 

$275 (DCAS costs) . 

^Calculated $43/hr x 8 hrs = $344 + $344 (8 hrs travel @ 
$43/hr) + $200 ( 2 days per diem @ $100/day) + $200 travel 
costs = $1,088 + $275 (DCAS costs). 

“Calculated $43/hr x 8 hrs = $344 + $688 (16 hrs travel @ 

$43/hr) + $300 ( 3 days per diem @ $100/day) + $575 travel 
costs = $1,907 + @275 (DCAS costs). 

“Calculated $43/hr x 4 hrs = $172 + $100 material handling + 
$325 test. 

“Calculated $43/hr x 8 hrs = $344 + $200 material handling + 
$650 test. 
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16 Calculated $43/hr x 24 hrs = $1,032 + $500 material 
handling + $800 test. 

17 Calculated DCAS 0 $34. 18/hr x 8 hrs = $275 + $480 
(procurement representative 0 $30/hr x 16 hrs) . 

18 Calculated procurement representative 0 $40/hr x 16 hrs. 
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APPENDIX E 



CLAUSES FOR FIXED PRICE — GREATEST VALUE PROCEDURES 



NOTICE TO PROSPECTIVE OFFERORS (NOV 1988) 

(a) This procurement is subject to a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
program. The test is authorized by the Assistant Secretary of 
the Navy (Shipbuilding and Logistics) for the acquisition of 
specific commodities within designated Federal Supply Classes 
(FSCs) by participating test activities. 

(b) The Government reserves the right to award to the 
contractor whose offer represents the best overall purchase 
value to the Government. As such, the basis for contract 
award will include an evaluation of proposed contractors' past 
quality performance history on the particular commodity or 
commodities, identified below, as recorded in the CES. 

(c) The procedures described in the clause of this 
solicitation entitled "ADDITIONAL EVALUATION FACTORS — TEST OF 
CONTRACTOR EVALUATION SYSTEM (NOV 1988)" will be used by the 
contracting officer to assist in determining the best purchase 
value for the Government — price, past quality performance, and 
other factors considered. 

(d) The commodities included in this test, as currently 
solicited, are: 

FSC No. FSC Nomenclature CLIN 
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ADDITIONAL EVALUATION FACTORS — TEST OF CONTRACTOR EVALUATION 
SYSTEM (NOV 1988) (FIXED PRICE-GREATEST VALUE PROCEDURES) 

(a) This procurement is part of a test of the Navy's 
Contractor Evaluation System (CES) "Red/Yellow/Green" (R/Y/G) 
Program, authorized by the Assistant Secretary of the Navy 
(Shipbuilding and Logistics) , for the acquisition of specific 
commodities by participating activities. At the end of the 
test, data concerning awards made during the period will be 
evaluated to assess the program's effectiveness and impact on 
the acquisition process. 

(b) The purpose of R/Y/G is to assist contracting personnel 
during source selection to determine the best value for the 
Government — price, past quality performance, and other factors 
considered. The test program uses accumulated contractor 
quality performance data to classify contractor's performance 
on select commodities as either "Red" (high risk) , "Yellow" 
(moderate risk), "Green" (low risk), or "Insufficient Data," 
based on the degree of risk to the Government of receiving 
poor quality products. A commodity's classification may 
change over time as new or revised quality performance data 
become available. 

(c) For purposes of source evaluation and selection, both the 
color classification of an offeror's commodity and the 
proposed price (s) shall be evaluated in accordance with 
weighted evaluation criteria established by the Government 
prior to the receipt of proposals. Price-related factors, 
such as transportation charges, discount terms, the cost of 
First Article Test, etc., will also be considered; however, 
no score or rating shall be applied. 

(d) Offerors are advised that, although price is of 
significance in determining the successful offeror, past 
quality performance on the proposed commodity (as classified 
with the R-Y-G database) is essentially more important, and 
shall be evaluated accordingly. Each of the R-Y-G 
classifications and its relative order of importance is 
summarized as follows: 

"Green"--Low risk. No extraordinary quality requirements or 
additional actions required; satisfactory quality history. 
Commodities within this classification are apportioned a 
greater weight or value in the evaluation than those 
classified as either "Yellow" or "Red." 

"Yellow"--Moderate risk. History of quality problems; special 
quality requirements/actions may be needed. Due to the 
additional quality assurance considerations that may be 
necessary, commodities within this classification are weighted 
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less than those classified as "Green", but are of greater 
value than those within the "Red" category. 

"Red"--High risk. Special alert to history of poor quality 
performance; contract award requires higher level approval. 
These commodities are apportioned the least available weight 
or value for past quality performance relative to commodities 
within the "Green" or "Yellow" classifications. 

"Insufficient Data"--Generally , may be commodities of first- 
time offerors or offerors for whom current, up-to-date quality 
performance history is unavailable; additional quality actions 
may be needed and invoked; however, commodities within this 
classification shall be evaluated solely on the basis of price 
and related factors. Past quality performance will not be a 
consideration in the evaluation of commodities for which 
current quality performance data is not set forth within the 
database . 

(e) Prospective offerors may address questions with regard to 
their assessment classification on particular commodities to: 
Naval Sea System Command Detachment, Navy Material Quality 
Assessment Office (NMQAO) , Federal Building, Room 423, 80 
Daniel Street, Portsmouth, NH 03801-3884, (Telephone) 608- 
431-9460. 
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APPENDIX F 



SAMPLE EVALUATION UNDER FIXED PRICE — GREATEST 
VALUE PROCEDURES 



Source Selection/Evaluation Method 



Total Points (Maximum) 


= 100 points ( 


%) 




Total Technical 


=60 points 


(%) 






Price 


=40 points 


(%) 






Classification 


Adjective 


Numerical 


. Ratine 


Green 


Excellent 




55-60 


points 


II 


Good 




40-54 


points 


Yellow 


Good 




35-39 


points 


II 


Average 




25-34 


points 


II 


Marginal 




16-24 


points 


Red 


Marginal 




10-15 


points 


ii 


Unacceptable 




0-14 


points 



Evaluation Measurements 



Yellow Classification 


Adi ective 


Numerical 


Ratine 


Reject Rate = 6%-8% 
(Code G) 


Good 


35-39 


points 


Reject Rate = 9%-ll% 
(Code G) 


Average 


25-34 


points 


Reject Rate = 12%-14% 
(Code G) 

3 Category "II" 


Marginal 


16-24 


points 


action QDRs 
(Code K) 

4 Category "II" 


Good 


35-39 


points 


action QDRs 
(Code K) 

5 Category "II" 


Average 


25-34 


points 


action QDRs 
(Code K) 


Marginal 


16-24 


points 



120 



LIST OF REFERENCES 



1. Office of the Assistant Secretary of the Navy (Shipbuilding 

and Logistics) Memorandum to Director, Defense Acquisition 
Regulatory Council, Subject: Navy Service Test of 

Red/Yellow/Green Concept under Contractor Evaluation System 
(CES) , 16 March 1989. 

2. Telephone conversaton between G. Elliott, Deputy Director, 
Naval Material Quality Assessment Office, Portsmouth, New 
Hampshire, and the author, 3 August 1989. 

3. Office of the Assistant Secretary of the Navy (Shipbuilding 

and Logistics) , Navy Product Deficiency Reporting and 
Evaluation Program, Volume 1, Contractor Evaluation Data 
Entry Guide , p. 1-1, Office of the Secretary of the Navy, 
Washington, D.C., 1988. 

4. Secretary of the Navy Instruction 4855.7, Subject: 
Department of the Navy Contractor Evaluation System, p. 1, 
28 March 1988. 

5. Secretary of the Navy Instruction 4855.3, Subject: Product 

Deficiency Reporting and Evaluation Program (PDREP) , p. 1, 
31 March 1987. 

6. Secretary of the Navy Instruction 4855.6, Subject: Navy 

Quality Deficiency Reporting Program, encl. (1), p. 1-1, 3 
February 1988. 

7. Telephone conversation between R. Morris, Program Analysis 
Officer, Naval Material Quality Assessment Office, 
Portsmouth, New Hampshire, and the author, 6 October 1989. 

8. Naval Supply Systems Command, Requirements Statement for 
Contractor Evaluation System for Naval Supply Systems 
Command . p. E-2, 27 May 1988. 

9. Contract Administration . V. 2, pp. 23-24, The Air Force 
Institute of Technology, School of Systems and Logistics, 
Wright-Patterson Air Force Base, Ohio, undated. 

10. Telephone conversation between S. DeWitt, Supervisory 
Quality Assurance Specialist, Naval Material Quality 
Assessment Office, Portsmouth, New Hampshire, and the 
author, 6 October 1989. 



121 



11. Contract Administration . V. 1, p. 62, The Air Force 
Institute of Technology, School of Systems and Logistics, 
Wright-Patterson Air Force Base, Ohio, undated. 

12. Interview between R. Morris, Program Analysis Officer, 
Naval Material Quality Assessment Office, Portsmouth, New 
Hampshire, and the author, 28 September 1989. 

13. Air Force Systems Command Regulation 800-54, Subject: 
Contractor Performance Assessment, p. 1, 11 August 1988. 

14. Telephone conversation between S. Nippert, Major, USAF, 
Procurement Staff Officer, Headquarters Air Force Systems 
Command, Andrews Air Force Base, Washington, D.C., and the 
author, 5 October 1989. 

15. Air Force Systems Command Form 125, Subject: Contractor 

Performance Assessment Report (CPAR) , May 1988. 

16. Demers, W.A., "Grading Contractor Performance," Military 
Forum , p. 41, May 1988. 

17. Zimmerman, S., "Low Bids No Longer Enough at Naval Avionics 

Center," Navy News and Undersea Technology , p. 8, 25 

January 1988. 

18. Telephone conversation between J. Wilson, Manager of 
Acquisition and Improvement, Naval Avionics Center, 
Indianapolis, Indiana, and the author, 31 July 1989. 

19. Naval Avionics Center Indianapolis Acquisition Information 

Memorandum (AIM 51) , Subject: Naval Avionics Center (NAC) 

Blue Ribbon Contractor Program, p. 1, 22 December 1987. 

20. Deming, W.E., Out of the Crisis , p. 1, Massachusetts 
Institute of Technology, Center for Advanced Engineering 
Study, 1986. 

21. Mann, N.R., The Keys to Excellence: The Story of the 

Deming Philosophy , p. 12, Prestwick Books, 1985. 

22. Ishikawa, K. , Guide to Quality Control , pp. 117-118, Asian 
Productivity Organization, JUSE Press, Ltd., 1968. 

23. Deming, W.E., Quality, Productivity, and Competitive 
Position , p. 25, Massachusetts Institute of Technology, 
Center for Advanced Engineering Study, 1982. 

24. Ishikawa, K. , What Is Total Quality Control? The Japanese 
Wav , p. 163, Prentice-Hall, Inc., 1985. 



122 



25. Stuelpnagel, T.R., "Total Quality Managment in Business — 
and Academia," Business Forum , p. 5, Fall 1988/Winter 1989. 

26. U.S. Department of Defense, Total Quality Management Master 
Plan, p. 1, August 1988. 

27. Interview between C. Gates, Captain, USN, Naval Air Systems 
Command, Washington, D.C., and the author, 25 September 
1989. 

28. U.S. Federal Acquisition Regulation, para. 9.405. 

29. Cibinic, J., Jr., and Nash, R.C., Administration of 
Government Contracts . 2d ed., 2d printing, p. 66, The 
George Washington University, 1986. 

30. Interview between S. Desbrow-Jensen , Assistant to the 
General Counsel, Office of General Counsel, Department of 
the Navy, Washington, D.C., and the author, 26 September 
1989. 

31. The Comptroller General of the United States Decision, B- 

224392.2, Matter of: B & W Service Industries, Inc., 2 

October 1986. 

32. The Comptroller General of the United States Decision, B- 

201095, Matter of: Utah Giophysical, Inc., 18 November 

1980. 

33. The Comptroller General of the United States Decision, B- 
192574, Matter of Electrospace Systems, Inc., 13 April 
1979 . 

34. The Comptroller General of the United States Decision, B- 

231607, Matter of: Sanford and Sons Company, 20 September 

1988 . 

35. The Comptroller General of the United States Decision, B- 

174012, Matter of: Siems International Electron Microscope 

Service, 25 July 1972. 

36. Interview between W. Mackinson, Assistant Deputy Commander 
Contracting Management, Naval Supply Systems Command, 
Washington, D.C., and the author, 25 September 1989. 

37. Naval Material Quality Assessment Office Report, Subject: 
Red/Yellow/Green, p. 9, September 1989. 

38. Telephone conversation between G. Kemp, Procurement 
Analyst, Naval Supply Systems Command, Washington, D.C., 
and the author, 16 August 1989. 



123 



39. Telephone conversation between D. Walsh, Commander, SC, 
USN, Naval Supply Center, Pensacola, Florida, and the 
author, 26 October 1989. 

40. Telephone conversation between R. Davidson, Deputy Director 
Purchase Division, Regional Contracting Department, Naval 
Supply Center, Charleston, South Carolina, and the author, 
31 October 1989. 

41. Telephone conversation between J. Minahan, Director of 
Hull, Mechanical, and Electrical Contracting Department, 
Navy Ships Parts Control Center, Mechanicsburg , 
Pennsylvania, and the author, 10 October 1989. 

42. Telephone conversation between J. Fackenthal, Supervisory 
Contract Specialist, Naval Air and Engineering Center, 
Lakehurst, New Jersey, and the author, 26 October 1989. 

43. Telephone conversation between J. Wilson, Manager of 

Acquisition and Improvement, Naval Avionics Center, 
Indianapolis, Indiana, and the author, 26 October 1989. 

44. Telephone conversation between L. Hargett, Quality 

Developmental Project Manager, Naval Aviation Depot, 
Pensacola, Florida, and the author, 27 October 1989. 

45. Telephone conversation between S. Cassell, Supervisory 

Quality Assurance Engineer, Naval Shipyard, Charleston, 
South Carolina, and the author, 31 October 1989. 

46. Telephone conversation between S. Jornov, Quality Assurance 
Specialist, Level 1/SS Section, Navy Ships Parts Control 
Center, Mechanicsburg, Pennsylvania, and the author, 6 
November 1989. 

47. Telephone conversation between R. Armitage, Quality 

Assurance Specialist, Procurement Division of Quality 

Assurance, Naval Air and Engineering Center, Lakehurst, New 
Jersey, and the author, 27 October 1989. 

48. Telephone conversation between R. Powell, Electronics 

Engineer, Quality Assurance, Naval Avionics Center, 
Indianapolis, Indiana, and the author, 27 October 1989. 



124 



INITIAL DISTRIBUTION LIST 



No. 



1. Defense Technical Information Center 
Cameron Station 

Alexandria, Virginia 22304 -6145 

2. Library, Code 0142 
Naval Postgraduate School 
Monterey, California 93943-5002 

3. Defense Logistics Studies Information 

Exchange 

U. S. Army Logistics Management Center 
Fort Lee, Virginia 23801 

4. Department Chairman, Code 54 
Department of Administrative Sciences 
Naval Postgraduate School 
Monterey, California 93943-5000 

5. Dr. D.V. Lamm, Code 54Lt 
Department of Administrative Sciences 
Naval Postgraduate School 
Monterey, California 93943-5000 

6. CDR E. Neil Hart, SC, USN, Code 54Hr 
Department of Administrative Sciences 
Naval Postgraduate School 
Monterey, California 93943-5000 

7. RADM W. H. Hauenstein, SC, USN 
Competition Advocate General of the Navy 
Crystal Plaza 5, Room 310 

2211 Jefferson Davis Highway 
Arlington, Virginia 22202 

8. Mr. W. Willoughby 

Office of the Assistant Secretary of 
the Navy 

(Shipbuilding and Logistics) 

Reliability, Maintainability, and 
Quality Assurance 
Washington, D.C. 20360-5000 



Copies 

2 

2 

2 

1 

5 

1 

2 



125 



9. Mr. P. Thompson 1 

Office of the Assistant Secretary 

of the Navy 

(Shipbuilding and Logistics) 

Reliability, Maintainability, and 
Quality Assurance 
Washington, D.C. 20360-5000 

10. Mr. W. Mackinson 1 

Assistant Deputy Commander for 

Contracting Management (NSUP-029) 

Naval Supply Systems Command 
Washington, D.C. 20376-5000 

11. Mr. J. Eaton 1 

Naval Material Quality Assessment 

Office 

Federal Building, Room 400 
80 Daniel Street 

Portsmouth, New Hampshire 03801-3884 

12. Mr. R. Morris 1 

Naval Material Quality Assessment 

Office 

Federal Building, Room 400 
80 Daniel Street 

Portsmouth, New Hampshire 03801-3884 

13. Ms. S. Desbrow-Jensen 1 

Assistant to the General Counsel 

Office of General Counsel 
Department of the Navy 
Washington, D.C. 20360-5110 

14. Mr. J. Minahan, Code 021 1 

Navy Ships Parts Control Center 

5450 Carlisle Pike 
Mechanicsburg, Pennsylvania 17055 

15. LT K . A . Hagmann , SC, USN 4 

3346 S. Wakefield Street, #A-1 

Arlington, Virginia 22206-1715 



[pi / - u ^ ^ 



126 



Thesis 
H1123 
c. 1 



Hagmann 

An evaluation of the 
Navy’s Red Yellow Green 
program and how this pro- 
gram is intended to im- 
prove the selection of 
quality cordacto rs 




